2023-01-24T18:46:22.743 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-24T18:46:22.744 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135617 --verbose --name yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi --owner scheduled_yuriw@teuthology --description rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} -- /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135617/orig.config.yaml 2023-01-24T18:46:22.774 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-24T18:46:22.847 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135617 branch: wip-yuri7-testing-2023-01-23-1532-quincy description: rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}} email: yweinstein@radhat.com first_in_suite: false job_id: '7135617' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri7-testing-2023-01-23-1532-quincy ceph: conf: global: mon election default strategy: 3 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 49f8fb05584886826e8eade75f7105fba754560c ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: 49f8fb05584886826e8eade75f7105fba754560c selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri7-testing-2023-01-23-1532-quincy sha1: 49f8fb05584886826e8eade75f7105fba754560c owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 - osd.8 - osd.9 - client.0 sha1: 49f8fb05584886826e8eade75f7105fba754560c sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri7-testing-2023-01-23-1532-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 49f8fb05584886826e8eade75f7105fba754560c targets: smithi121.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJNlc8wQwRp5m7j/w2nuuTbMzSNL1oHGr/aqbttTjsH3t0nC5ZaKr7kp9YVzV0OCBoOzqXm+JLDX7DI/ykEzyc= tasks: - install: null - ceph: conf: osd: osd max object name len: 400 osd max object namespace len: 64 log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - workunit: clients: all: - rados/test_health_warnings.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_16:05:13 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-24T18:46:22.848 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa; will attempt to use it 2023-01-24T18:46:22.848 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa/tasks 2023-01-24T18:46:22.849 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-24T18:46:22.851 INFO:teuthology.task.internal:Checking packages... 2023-01-24T18:46:22.889 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '49f8fb05584886826e8eade75f7105fba754560c' 2023-01-24T18:46:22.890 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-24T18:46:22.891 INFO:teuthology.packaging:ref: None 2023-01-24T18:46:22.891 INFO:teuthology.packaging:tag: None 2023-01-24T18:46:22.891 INFO:teuthology.packaging:branch: wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:46:22.891 INFO:teuthology.packaging:sha1: 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:46:22.892 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:46:23.046 INFO:teuthology.task.internal:Found packages for ceph version 17.2.5-727.g49f8fb05 2023-01-24T18:46:23.046 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-24T18:46:23.058 INFO:teuthology.task.internal:no buildpackages task found 2023-01-24T18:46:23.058 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-24T18:46:23.122 INFO:teuthology.task.internal:Saving configuration 2023-01-24T18:46:23.139 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-24T18:46:23.188 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-24T18:46:23.217 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi121.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135617', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-24 18:41:40.789467', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJNlc8wQwRp5m7j/w2nuuTbMzSNL1oHGr/aqbttTjsH3t0nC5ZaKr7kp9YVzV0OCBoOzqXm+JLDX7DI/ykEzyc='} 2023-01-24T18:46:23.218 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-24T18:46:23.255 INFO:teuthology.task.internal:roles: ubuntu@smithi121.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0'] 2023-01-24T18:46:23.255 INFO:teuthology.run_tasks:Running task console_log... 2023-01-24T18:46:23.332 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f53d15f55e0>, signals=[15]) 2023-01-24T18:46:23.333 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-24T18:46:23.379 INFO:teuthology.task.internal:Opening connections... 2023-01-24T18:46:23.380 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:46:23.381 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:46:23.448 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-24T18:46:23.460 DEBUG:teuthology.orchestra.run.smithi121:> uname -m 2023-01-24T18:46:23.475 INFO:teuthology.orchestra.run.smithi121.stdout:x86_64 2023-01-24T18:46:23.476 DEBUG:teuthology.orchestra.run.smithi121:> cat /etc/os-release 2023-01-24T18:46:23.527 INFO:teuthology.orchestra.run.smithi121.stdout:NAME="Red Hat Enterprise Linux" 2023-01-24T18:46:23.528 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION="8.4 (Ootpa)" 2023-01-24T18:46:23.528 INFO:teuthology.orchestra.run.smithi121.stdout:ID="rhel" 2023-01-24T18:46:23.529 INFO:teuthology.orchestra.run.smithi121.stdout:ID_LIKE="fedora" 2023-01-24T18:46:23.529 INFO:teuthology.orchestra.run.smithi121.stdout:VERSION_ID="8.4" 2023-01-24T18:46:23.530 INFO:teuthology.orchestra.run.smithi121.stdout:PLATFORM_ID="platform:el8" 2023-01-24T18:46:23.531 INFO:teuthology.orchestra.run.smithi121.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-01-24T18:46:23.531 INFO:teuthology.orchestra.run.smithi121.stdout:ANSI_COLOR="0;31" 2023-01-24T18:46:23.531 INFO:teuthology.orchestra.run.smithi121.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-01-24T18:46:23.531 INFO:teuthology.orchestra.run.smithi121.stdout:HOME_URL="https://www.redhat.com/" 2023-01-24T18:46:23.532 INFO:teuthology.orchestra.run.smithi121.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-24T18:46:23.532 INFO:teuthology.orchestra.run.smithi121.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-24T18:46:23.532 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:46:23.533 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-24T18:46:23.533 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-01-24T18:46:23.533 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-24T18:46:23.533 INFO:teuthology.orchestra.run.smithi121.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-01-24T18:46:23.534 INFO:teuthology.lock.ops:Updating smithi121.front.sepia.ceph.com on lock server 2023-01-24T18:46:23.560 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-24T18:46:23.573 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-24T18:46:23.599 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-24T18:46:23.599 DEBUG:teuthology.orchestra.run.smithi121:> test '!' -e /home/ubuntu/cephtest 2023-01-24T18:46:23.614 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-24T18:46:23.697 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-24T18:46:23.697 DEBUG:teuthology.orchestra.run.smithi121:> test -z $(ls -A /var/lib/ceph) 2023-01-24T18:46:23.713 INFO:teuthology.orchestra.run.smithi121.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-01-24T18:46:23.714 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-24T18:46:23.757 INFO:teuthology.run_tasks:Running task kernel... 2023-01-24T18:46:23.789 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-24T18:46:23.790 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-24T18:46:23.790 DEBUG:teuthology.orchestra.run.smithi121:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-24T18:46:23.806 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:46:23.807 DEBUG:teuthology.orchestra.run.smithi121:> uname -r 2023-01-24T18:46:23.861 INFO:teuthology.orchestra.run.smithi121.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:46:23.862 INFO:teuthology.task.kernel:Running kernel on smithi121: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:46:23.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum install -y kernel 2023-01-24T18:46:25.359 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:46:25.360 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-24T18:46:26.702 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 35 kB/s | 3.5 kB 00:00 2023-01-24T18:46:28.086 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 13 MB 00:01 2023-01-24T18:46:30.954 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2023-01-24T18:46:31.659 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-01-24T18:46:31.660 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-01-24T18:46:31.660 INFO:teuthology.orchestra.run.smithi121.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-01-24T18:46:31.790 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:46:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:46:31.791 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:46:31.896 DEBUG:teuthology.orchestra.run.smithi121:> echo no | sudo yum reinstall kernel || true 2023-01-24T18:46:32.239 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:46:32.240 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-24T18:46:32.519 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:02 ago on Tue 24 Jan 2023 06:46:30 PM UTC. 2023-01-24T18:46:32.657 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:46:32.658 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:46:32.658 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:46:32.690 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2023-01-24T18:46:32.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum reinstall -y kernel || true 2023-01-24T18:46:33.083 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:46:33.084 INFO:teuthology.orchestra.run.smithi121.stdout:Unable to read consumer identity 2023-01-24T18:46:33.359 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:03 ago on Tue 24 Jan 2023 06:46:30 PM UTC. 2023-01-24T18:46:33.515 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-01-24T18:46:33.515 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:46:33.515 INFO:teuthology.orchestra.run.smithi121.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-01-24T18:46:33.533 INFO:teuthology.orchestra.run.smithi121.stderr:Error: No packages marked for reinstall. 2023-01-24T18:46:33.589 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q kernel --last | head -n 1 2023-01-24T18:46:33.713 INFO:teuthology.orchestra.run.smithi121.stdout:kernel-4.18.0-305.25.1.el8_4.x86_64 Wed 21 Dec 2022 01:05:24 PM UTC 2023-01-24T18:46:33.715 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.25.1.el8_4.x86_64 2023-01-24T18:46:33.715 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-24T18:46:33.716 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-24T18:46:33.716 DEBUG:teuthology.orchestra.run.smithi121:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-24T18:46:33.747 INFO:teuthology.orchestra.run.smithi121.stdout:ttyS1 2023-01-24T18:46:33.763 DEBUG:teuthology.parallel:result is None 2023-01-24T18:46:33.763 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-24T18:46:33.775 INFO:teuthology.task.internal:Creating test directory... 2023-01-24T18:46:33.775 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-24T18:46:33.817 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-24T18:46:33.828 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-24T18:46:33.894 INFO:teuthology.task.internal:Creating archive directory... 2023-01-24T18:46:33.895 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-24T18:46:33.917 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-24T18:46:33.928 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-24T18:46:33.928 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-24T18:46:33.985 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:46:33.994 INFO:teuthology.orchestra.run.smithi121.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-24T18:46:33.996 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-24T18:46:34.007 INFO:teuthology.task.internal:Configuring sudo... 2023-01-24T18:46:34.007 DEBUG:teuthology.orchestra.run.smithi121:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-24T18:46:34.060 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-24T18:46:34.073 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-24T18:46:34.073 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-24T18:46:34.114 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:46:34.192 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-24T18:46:34.234 DEBUG:teuthology.orchestra.run.smithi121:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:46:34.311 DEBUG:teuthology.orchestra.run.smithi121:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-24T18:46:34.353 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:46:34.354 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-24T18:46:34.416 DEBUG:teuthology.orchestra.run.smithi121:> sudo service rsyslog restart 2023-01-24T18:46:34.501 INFO:teuthology.orchestra.run.smithi121.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-24T18:46:34.517 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-24T18:46:34.525 INFO:teuthology.task.internal:Starting timer... 2023-01-24T18:46:34.526 INFO:teuthology.run_tasks:Running task pcp... 2023-01-24T18:46:34.598 INFO:teuthology.run_tasks:Running task selinux... 2023-01-24T18:46:34.657 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-24T18:46:34.658 DEBUG:teuthology.orchestra.run.smithi121:> sudo service auditd rotate 2023-01-24T18:46:34.725 INFO:teuthology.orchestra.run.smithi121.stdout:Rotating logs: [ OK ] 2023-01-24T18:46:34.727 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-24T18:46:34.727 DEBUG:teuthology.orchestra.run.smithi121:> /usr/sbin/getenforce 2023-01-24T18:46:34.750 INFO:teuthology.orchestra.run.smithi121.stdout:Permissive 2023-01-24T18:46:34.751 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi121.front.sepia.ceph.com': 'permissive'} 2023-01-24T18:46:34.751 DEBUG:teuthology.orchestra.run.smithi121:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-24T18:46:34.814 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:46:34.815 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-24T18:46:34.816 DEBUG:teuthology.orchestra.run.smithi121:> sudo /usr/sbin/setenforce permissive 2023-01-24T18:46:34.882 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-24T18:46:34.893 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-01-24T18:46:34.898 INFO:teuthology.repo_utils:Fetching main from origin 2023-01-24T18:46:35.036 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-24T18:46:35.054 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-24T18:46:35.055 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi121.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-24T18:53:02.112 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi121.front.sepia.ceph.com')] 2023-01-24T18:53:02.114 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi121.front.sepia.ceph.com' 2023-01-24T18:53:02.115 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi121.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-24T18:53:02.186 DEBUG:teuthology.orchestra.run.smithi121:> true 2023-01-24T18:53:02.251 INFO:teuthology.run_tasks:Running task clock... 2023-01-24T18:53:02.262 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-24T18:53:02.263 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-24T18:53:02.264 DEBUG:teuthology.orchestra.run.smithi121:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-24T18:53:02.321 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-24T18:53:02.334 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-24T18:53:02.368 INFO:teuthology.orchestra.run.smithi121.stderr:sudo: ntpd: command not found 2023-01-24T18:53:02.377 INFO:teuthology.orchestra.run.smithi121.stdout:506 Cannot talk to daemon 2023-01-24T18:53:02.390 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-24T18:53:02.403 INFO:teuthology.orchestra.run.smithi121.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-24T18:53:02.471 INFO:teuthology.orchestra.run.smithi121.stderr:bash: ntpq: command not found 2023-01-24T18:53:02.474 INFO:teuthology.orchestra.run.smithi121.stdout:210 Number of sources = 4 2023-01-24T18:53:02.475 INFO:teuthology.orchestra.run.smithi121.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-24T18:53:02.475 INFO:teuthology.orchestra.run.smithi121.stdout:=============================================================================== 2023-01-24T18:53:02.475 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:53:02.475 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:53:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:53:02.476 INFO:teuthology.orchestra.run.smithi121.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-24T18:53:02.476 INFO:teuthology.run_tasks:Running task install... 2023-01-24T18:53:02.487 DEBUG:teuthology.task.install:project ceph 2023-01-24T18:53:02.488 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}} 2023-01-24T18:53:02.489 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'} 2023-01-24T18:53:02.489 INFO:teuthology.task.install:Using flavor: default 2023-01-24T18:53:02.495 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-24T18:53:02.495 INFO:teuthology.task.install:extra packages: [] 2023-01-24T18:53:02.496 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-24T18:53:02.553 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-24T18:53:02.553 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:53:02.554 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'tag': None, 'wait_for_package': False} 2023-01-24T18:53:02.555 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-24T18:53:02.630 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-24T18:53:02.631 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:53:02.632 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:53:02.789 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/ 2023-01-24T18:53:02.790 INFO:teuthology.task.install.rpm:Package version is 17.2.5-727.g49f8fb05 2023-01-24T18:53:02.931 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri7-testing-2023-01-23-1532-quincy/49f8fb05584886826e8eade75f7105fba754560c/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-24T18:53:02.932 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:53:02.932 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-24T18:53:02.957 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-24T18:53:02.957 DEBUG:teuthology.orchestra.run.smithi121:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/49f8fb05584886826e8eade75f7105fba754560c/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-24T18:53:03.021 DEBUG:teuthology.orchestra.run.smithi121:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-24T18:53:03.094 DEBUG:teuthology.orchestra.run.smithi121:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-24T18:53:03.158 INFO:teuthology.orchestra.run.smithi121.stdout:check_obsoletes = 1 2023-01-24T18:53:03.159 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum clean all 2023-01-24T18:53:03.542 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:53:03.812 INFO:teuthology.orchestra.run.smithi121.stdout:44 files removed 2023-01-24T18:53:03.862 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-radosgw 2023-01-24T18:53:04.206 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:53:04.954 INFO:teuthology.orchestra.run.smithi121.stdout:ceph packages for x86_64 323 kB/s | 78 kB 00:00 2023-01-24T18:53:05.195 INFO:teuthology.orchestra.run.smithi121.stdout:ceph noarch packages 46 kB/s | 9.2 kB 00:00 2023-01-24T18:53:05.427 INFO:teuthology.orchestra.run.smithi121.stdout:ceph source packages 7.2 kB/s | 1.5 kB 00:00 2023-01-24T18:53:05.549 INFO:teuthology.orchestra.run.smithi121.stdout:Copr repo for python3-asyncssh owned by ceph 37 kB/s | 3.5 kB 00:00 2023-01-24T18:53:06.431 INFO:teuthology.orchestra.run.smithi121.stdout:Extra Packages for Enterprise Linux 16 MB/s | 13 MB 00:00 2023-01-24T18:53:09.245 INFO:teuthology.orchestra.run.smithi121.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2023-01-24T18:53:09.608 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 134 MB/s | 38 MB 00:00 2023-01-24T18:53:15.110 INFO:teuthology.orchestra.run.smithi121.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 128 MB/s | 33 MB 00:00 2023-01-24T18:53:23.408 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:53:23.411 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-24T18:53:23.411 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:53:23.411 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-24T18:53:23.411 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:53:23.412 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:53:23.412 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrading: 2023-01-24T18:53:23.412 INFO:teuthology.orchestra.run.smithi121.stdout: librados2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:53:23.412 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.6 M 2023-01-24T18:53:23.413 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:53:23.413 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.1 M 2023-01-24T18:53:23.413 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 23 M 2023-01-24T18:53:23.413 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 24 k 2023-01-24T18:53:23.413 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-24T18:53:23.414 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 793 k 2023-01-24T18:53:23.414 INFO:teuthology.orchestra.run.smithi121.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-24T18:53:23.414 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-24T18:53:23.414 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 506 k 2023-01-24T18:53:23.415 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-01-24T18:53:23.415 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2 x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 5.5 M 2023-01-24T18:53:23.415 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-24T18:53:23.415 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-24T18:53:23.418 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 45 k 2023-01-24T18:53:23.418 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 108 k 2023-01-24T18:53:23.418 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 190 k 2023-01-24T18:53:23.418 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-24T18:53:23.419 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 370 k 2023-01-24T18:53:23.419 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 357 k 2023-01-24T18:53:23.419 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 107 k 2023-01-24T18:53:23.419 INFO:teuthology.orchestra.run.smithi121.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-24T18:53:23.419 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:53:23.420 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:53:23.420 INFO:teuthology.orchestra.run.smithi121.stdout:=================================================================================================== 2023-01-24T18:53:23.420 INFO:teuthology.orchestra.run.smithi121.stdout:Install 21 Packages 2023-01-24T18:53:23.421 INFO:teuthology.orchestra.run.smithi121.stdout:Upgrade 2 Packages 2023-01-24T18:53:23.421 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:53:23.422 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 63 M 2023-01-24T18:53:23.422 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:53:24.187 INFO:teuthology.orchestra.run.smithi121.stdout:(1/23): ceph-base-17.2.5-727.g49f8fb05.el8.x86_ 6.7 MB/s | 5.1 MB 00:00 2023-01-24T18:53:24.278 INFO:teuthology.orchestra.run.smithi121.stdout:(2/23): ceph-selinux-17.2.5-727.g49f8fb05.el8.x 266 kB/s | 24 kB 00:00 2023-01-24T18:53:24.417 INFO:teuthology.orchestra.run.smithi121.stdout:(3/23): libcephfs2-17.2.5-727.g49f8fb05.el8.x86 5.6 MB/s | 793 kB 00:00 2023-01-24T18:53:24.539 INFO:teuthology.orchestra.run.smithi121.stdout:(4/23): libradosstriper1-17.2.5-727.g49f8fb05.e 4.1 MB/s | 506 kB 00:00 2023-01-24T18:53:25.346 INFO:teuthology.orchestra.run.smithi121.stdout:(5/23): ceph-radosgw-17.2.5-727.g49f8fb05.el8.x 8.6 MB/s | 17 MB 00:01 2023-01-24T18:53:25.366 INFO:teuthology.orchestra.run.smithi121.stdout:(6/23): librgw2-17.2.5-727.g49f8fb05.el8.x86_64 6.7 MB/s | 5.5 MB 00:00 2023-01-24T18:53:25.375 INFO:teuthology.orchestra.run.smithi121.stdout:(7/23): python3-ceph-argparse-17.2.5-727.g49f8f 1.5 MB/s | 45 kB 00:00 2023-01-24T18:53:25.391 INFO:teuthology.orchestra.run.smithi121.stdout:(8/23): python3-ceph-common-17.2.5-727.g49f8fb0 4.3 MB/s | 108 kB 00:00 2023-01-24T18:53:25.409 INFO:teuthology.orchestra.run.smithi121.stdout:(9/23): python3-cephfs-17.2.5-727.g49f8fb05.el8 5.5 MB/s | 190 kB 00:00 2023-01-24T18:53:25.433 INFO:teuthology.orchestra.run.smithi121.stdout:(10/23): python3-rados-17.2.5-727.g49f8fb05.el8 8.7 MB/s | 370 kB 00:00 2023-01-24T18:53:25.458 INFO:teuthology.orchestra.run.smithi121.stdout:(11/23): python3-rbd-17.2.5-727.g49f8fb05.el8.x 7.1 MB/s | 357 kB 00:00 2023-01-24T18:53:25.472 INFO:teuthology.orchestra.run.smithi121.stdout:(12/23): python3-rgw-17.2.5-727.g49f8fb05.el8.x 2.7 MB/s | 107 kB 00:00 2023-01-24T18:53:25.705 INFO:teuthology.orchestra.run.smithi121.stdout:(13/23): ceph-common-17.2.5-727.g49f8fb05.el8.x 10 MB/s | 23 MB 00:02 2023-01-24T18:53:25.706 INFO:teuthology.orchestra.run.smithi121.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 251 kB/s | 59 kB 00:00 2023-01-24T18:53:25.735 INFO:teuthology.orchestra.run.smithi121.stdout:(15/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.1 MB/s | 306 kB 00:00 2023-01-24T18:53:25.762 INFO:teuthology.orchestra.run.smithi121.stdout:(16/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.7 MB/s | 47 kB 00:00 2023-01-24T18:53:25.774 INFO:teuthology.orchestra.run.smithi121.stdout:(17/23): python3-prettytable-0.7.2-14.el8.noarc 3.7 MB/s | 44 kB 00:00 2023-01-24T18:53:25.787 INFO:teuthology.orchestra.run.smithi121.stdout:(18/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 19 MB/s | 259 kB 00:00 2023-01-24T18:53:25.803 INFO:teuthology.orchestra.run.smithi121.stdout:(19/23): librdkafka-0.11.4-1.el8.x86_64.rpm 22 MB/s | 353 kB 00:00 2023-01-24T18:53:25.816 INFO:teuthology.orchestra.run.smithi121.stdout:(20/23): libunwind-1.3.1-3.el8.x86_64.rpm 678 kB/s | 75 kB 00:00 2023-01-24T18:53:25.905 INFO:teuthology.orchestra.run.smithi121.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 8.8 MB/s | 1.7 MB 00:00 2023-01-24T18:53:26.040 INFO:teuthology.orchestra.run.smithi121.stdout:(22/23): librbd1-17.2.5-727.g49f8fb05.el8.x86_6 16 MB/s | 3.6 MB 00:00 2023-01-24T18:53:26.095 INFO:teuthology.orchestra.run.smithi121.stdout:(23/23): librados2-17.2.5-727.g49f8fb05.el8.x86 12 MB/s | 3.5 MB 00:00 2023-01-24T18:53:26.097 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:53:26.098 INFO:teuthology.orchestra.run.smithi121.stdout:Total 23 MB/s | 63 MB 00:02 2023-01-24T18:53:26.101 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:53:26.145 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:53:26.145 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:53:26.911 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:53:26.915 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:53:28.006 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:53:28.236 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:53:28.623 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2023-01-24T18:53:28.836 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:53:28.938 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:53:29.134 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:53:29.419 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2023-01-24T18:53:29.636 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:53:30.064 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:53:30.286 INFO:teuthology.orchestra.run.smithi121.stdout: Upgrading : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:53:30.386 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:53:30.578 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:53:30.645 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 7/25 2023-01-24T18:53:30.997 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 8/25 2023-01-24T18:53:31.684 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2023-01-24T18:53:31.844 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:53:31.908 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:53:32.038 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 11/25 2023-01-24T18:53:32.247 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:53:32.488 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:53:32.747 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 14/25 2023-01-24T18:53:32.894 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:53:32.963 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 15/25 2023-01-24T18:53:33.179 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2023-01-24T18:53:33.362 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2023-01-24T18:53:33.624 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2023-01-24T18:53:33.819 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 19/25 2023-01-24T18:53:38.470 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:53:38.701 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:53:39.266 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 20/25 2023-01-24T18:53:39.563 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:53:39.646 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 21/25 2023-01-24T18:53:39.767 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:53:57.388 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:53:57.388 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /sys 2023-01-24T18:53:57.388 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /proc 2023-01-24T18:53:57.389 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-01-24T18:53:57.389 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /run 2023-01-24T18:53:57.389 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /mnt 2023-01-24T18:53:57.390 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /var/tmp 2023-01-24T18:53:57.390 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /home 2023-01-24T18:53:57.390 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /tmp 2023-01-24T18:53:57.391 INFO:teuthology.orchestra.run.smithi121.stdout:skipping the directory /dev 2023-01-24T18:53:57.391 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:53:59.075 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:53:59.175 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 23/25 2023-01-24T18:53:59.176 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:53:59.382 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-24T18:53:59.382 INFO:teuthology.orchestra.run.smithi121.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:53:59.901 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:53:59.902 INFO:teuthology.orchestra.run.smithi121.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-24T18:53:59.902 INFO:teuthology.orchestra.run.smithi121.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-24T18:53:59.902 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:00.158 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 1/25 2023-01-24T18:54:00.159 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2/25 2023-01-24T18:54:00.159 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 3/25 2023-01-24T18:54:00.159 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 4/25 2023-01-24T18:54:00.159 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 5/25 2023-01-24T18:54:00.159 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 6/25 2023-01-24T18:54:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 7/25 2023-01-24T18:54:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x 8/25 2023-01-24T18:54:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86 9/25 2023-01-24T18:54:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 10/25 2023-01-24T18:54:00.160 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 11/25 2023-01-24T18:54:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 12/25 2023-01-24T18:54:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 13/25 2023-01-24T18:54:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-24T18:54:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-24T18:54:00.161 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-24T18:54:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-24T18:54:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-24T18:54:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 19/25 2023-01-24T18:54:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 20/25 2023-01-24T18:54:00.162 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 21/25 2023-01-24T18:54:00.163 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 22/25 2023-01-24T18:54:00.163 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-24T18:54:00.163 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 24/25 2023-01-24T18:54:01.486 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-24T18:54:01.486 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:47 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:03.413 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:54:03.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:03.528 INFO:teuthology.orchestra.run.smithi121.stdout:Upgraded: 2023-01-24T18:54:03.529 INFO:teuthology.orchestra.run.smithi121.stdout: librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.529 INFO:teuthology.orchestra.run.smithi121.stdout: librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.529 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:54:03.529 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.530 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.530 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-radosgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.531 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-selinux-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.531 INFO:teuthology.orchestra.run.smithi121.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-24T18:54:03.532 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.532 INFO:teuthology.orchestra.run.smithi121.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-24T18:54:03.532 INFO:teuthology.orchestra.run.smithi121.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-24T18:54:03.532 INFO:teuthology.orchestra.run.smithi121.stdout: libradosstriper1-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-01-24T18:54:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: librgw2-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-24T18:54:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-24T18:54:03.533 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-argparse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.534 INFO:teuthology.orchestra.run.smithi121.stdout: python3-ceph-common-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.534 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.534 INFO:teuthology.orchestra.run.smithi121.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-24T18:54:03.534 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.534 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.535 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:03.535 INFO:teuthology.orchestra.run.smithi121.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-24T18:54:03.535 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:03.535 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:54:03.680 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-test 2023-01-24T18:54:04.040 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:54:04.770 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:00:50 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:06.557 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:54:06.558 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================== 2023-01-24T18:54:06.558 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:54:06.558 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================== 2023-01-24T18:54:06.559 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:54:06.559 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 52 M 2023-01-24T18:54:06.559 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:54:06.559 INFO:teuthology.orchestra.run.smithi121.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-01-24T18:54:06.559 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 174 k 2023-01-24T18:54:06.560 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-24T18:54:06.560 INFO:teuthology.orchestra.run.smithi121.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-01-24T18:54:06.560 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-24T18:54:06.560 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:06.560 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:54:06.561 INFO:teuthology.orchestra.run.smithi121.stdout:=========================================================================================== 2023-01-24T18:54:06.561 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2023-01-24T18:54:06.561 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:06.562 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 53 M 2023-01-24T18:54:06.562 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 233 M 2023-01-24T18:54:06.562 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:54:06.563 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 20 MB/s | 70 kB 00:00 2023-01-24T18:54:06.595 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): jq-1.5-12.el8.x86_64.rpm 5.1 MB/s | 161 kB 00:00 2023-01-24T18:54:06.616 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 8.9 MB/s | 187 kB 00:00 2023-01-24T18:54:06.630 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): socat-1.7.3.3-2.el8.x86_64.rpm 21 MB/s | 302 kB 00:00 2023-01-24T18:54:06.895 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): libcephsqlite-17.2.5-727.g49f8fb05.el8.x 519 kB/s | 174 kB 00:00 2023-01-24T18:54:13.213 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): ceph-test-17.2.5-727.g49f8fb05.el8.x86_6 7.9 MB/s | 52 MB 00:06 2023-01-24T18:54:13.216 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:54:13.217 INFO:teuthology.orchestra.run.smithi121.stdout:Total 8.0 MB/s | 53 MB 00:06 2023-01-24T18:54:13.220 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:54:13.243 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:54:13.243 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:54:13.580 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:54:13.584 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:54:14.096 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:54:14.334 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 1/6 2023-01-24T18:54:14.526 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:54:14.590 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 2/6 2023-01-24T18:54:14.749 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : jq-1.5-12.el8.x86_64 3/6 2023-01-24T18:54:14.950 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-24T18:54:15.085 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:54:20.059 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 5/6 2023-01-24T18:54:20.315 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:54:20.586 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 6/6 2023-01-24T18:54:20.586 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 1/6 2023-01-24T18:54:20.587 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2/6 2023-01-24T18:54:20.587 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-24T18:54:20.587 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : jq-1.5-12.el8.x86_64 4/6 2023-01-24T18:54:20.587 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-24T18:54:21.585 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 6/6 2023-01-24T18:54:21.588 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:07 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:23.427 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:54:23.523 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:23.524 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:54:23.524 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-test-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:23.525 INFO:teuthology.orchestra.run.smithi121.stdout: jq-1.5-12.el8.x86_64 2023-01-24T18:54:23.525 INFO:teuthology.orchestra.run.smithi121.stdout: libcephsqlite-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:23.525 INFO:teuthology.orchestra.run.smithi121.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-24T18:54:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: socat-1.7.3.3-2.el8.x86_64 2023-01-24T18:54:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-24T18:54:23.526 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:23.527 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:54:23.636 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph 2023-01-24T18:54:23.983 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:54:24.732 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:10 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:26.542 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:54:26.545 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================== 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================== 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout: ceph x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 6.4 k 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:54:26.546 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 2.3 M 2023-01-24T18:54:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 1.6 M 2023-01-24T18:54:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 233 k 2023-01-24T18:54:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.9 M 2023-01-24T18:54:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 17 M 2023-01-24T18:54:26.547 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-24T18:54:26.548 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-24T18:54:26.548 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-24T18:54:26.548 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-24T18:54:26.548 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-24T18:54:26.549 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-24T18:54:26.550 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-24T18:54:26.550 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-24T18:54:26.550 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-24T18:54:26.550 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-24T18:54:26.550 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-24T18:54:26.551 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-24T18:54:26.551 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-24T18:54:26.551 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-24T18:54:26.551 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-24T18:54:26.551 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-24T18:54:26.552 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-24T18:54:26.552 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:26.552 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:54:26.552 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================== 2023-01-24T18:54:26.552 INFO:teuthology.orchestra.run.smithi121.stdout:Install 28 Packages 2023-01-24T18:54:26.553 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:26.553 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 27 M 2023-01-24T18:54:26.553 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 105 M 2023-01-24T18:54:26.554 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:54:26.649 INFO:teuthology.orchestra.run.smithi121.stdout:(1/28): ceph-17.2.5-727.g49f8fb05.el8.x86_64.rp 65 kB/s | 6.4 kB 00:00 2023-01-24T18:54:26.907 INFO:teuthology.orchestra.run.smithi121.stdout:(2/28): ceph-mgr-17.2.5-727.g49f8fb05.el8.x86_6 4.4 MB/s | 1.6 MB 00:00 2023-01-24T18:54:26.929 INFO:teuthology.orchestra.run.smithi121.stdout:(3/28): ceph-mds-17.2.5-727.g49f8fb05.el8.x86_6 6.0 MB/s | 2.3 MB 00:00 2023-01-24T18:54:26.967 INFO:teuthology.orchestra.run.smithi121.stdout:(4/28): ceph-mgr-modules-core-17.2.5-727.g49f8f 6.0 MB/s | 233 kB 00:00 2023-01-24T18:54:27.097 INFO:teuthology.orchestra.run.smithi121.stdout:(5/28): ceph-mon-17.2.5-727.g49f8fb05.el8.x86_6 8.8 MB/s | 3.9 MB 00:00 2023-01-24T18:54:27.122 INFO:teuthology.orchestra.run.smithi121.stdout:(6/28): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 282 kB/s | 44 kB 00:00 2023-01-24T18:54:27.256 INFO:teuthology.orchestra.run.smithi121.stdout:(7/28): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.3 MB/s | 173 kB 00:00 2023-01-24T18:54:27.270 INFO:teuthology.orchestra.run.smithi121.stdout:(8/28): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.0 MB/s | 185 kB 00:00 2023-01-24T18:54:27.310 INFO:teuthology.orchestra.run.smithi121.stdout:(9/28): python3-jaraco-6.2-6.el8.noarch.rpm 278 kB/s | 11 kB 00:00 2023-01-24T18:54:27.349 INFO:teuthology.orchestra.run.smithi121.stdout:(10/28): python3-jaraco-functools-2.0-4.el8.noa 462 kB/s | 18 kB 00:00 2023-01-24T18:54:27.383 INFO:teuthology.orchestra.run.smithi121.stdout:(11/28): python3-logutils-0.3.5-11.el8.noarch.r 1.4 MB/s | 49 kB 00:00 2023-01-24T18:54:27.419 INFO:teuthology.orchestra.run.smithi121.stdout:(12/28): python3-more-itertools-7.2.0-3.el8.noa 1.6 MB/s | 59 kB 00:00 2023-01-24T18:54:27.470 INFO:teuthology.orchestra.run.smithi121.stdout:(13/28): python3-cherrypy-18.4.0-1.el8.noarch.r 1.8 MB/s | 384 kB 00:00 2023-01-24T18:54:27.514 INFO:teuthology.orchestra.run.smithi121.stdout:(14/28): python3-portend-2.6-1.el8.noarch.rpm 354 kB/s | 16 kB 00:00 2023-01-24T18:54:27.545 INFO:teuthology.orchestra.run.smithi121.stdout:(15/28): python3-pecan-1.3.2-9.el8.noarch.rpm 2.2 MB/s | 283 kB 00:00 2023-01-24T18:54:27.566 INFO:teuthology.orchestra.run.smithi121.stdout:(16/28): python3-simplegeneric-0.8.1-17.el8.noa 365 kB/s | 19 kB 00:00 2023-01-24T18:54:27.600 INFO:teuthology.orchestra.run.smithi121.stdout:(17/28): python3-singledispatch-3.4.0.3-18.el8. 434 kB/s | 24 kB 00:00 2023-01-24T18:54:27.610 INFO:teuthology.orchestra.run.smithi121.stdout:(18/28): python3-tempora-1.14.1-5.el8.noarch.rp 660 kB/s | 29 kB 00:00 2023-01-24T18:54:27.743 INFO:teuthology.orchestra.run.smithi121.stdout:(19/28): ceph-osd-17.2.5-727.g49f8fb05.el8.x86_ 20 MB/s | 17 MB 00:00 2023-01-24T18:54:27.743 INFO:teuthology.orchestra.run.smithi121.stdout:(20/28): python3-trustme-0.6.0-4.el8.noarch.rpm 192 kB/s | 27 kB 00:00 2023-01-24T18:54:27.772 INFO:teuthology.orchestra.run.smithi121.stdout:(21/28): python3-webtest-2.0.33-1.el8.noarch.rp 2.9 MB/s | 85 kB 00:00 2023-01-24T18:54:27.796 INFO:teuthology.orchestra.run.smithi121.stdout:(22/28): python3-waitress-1.2.1-2.el8.1.noarch. 1.3 MB/s | 241 kB 00:00 2023-01-24T18:54:27.800 INFO:teuthology.orchestra.run.smithi121.stdout:(23/28): python3-zc-lockfile-2.0-2.el8.noarch.r 834 kB/s | 23 kB 00:00 2023-01-24T18:54:27.831 INFO:teuthology.orchestra.run.smithi121.stdout:(24/28): python3-werkzeug-0.12.2-4.el8.noarch.r 13 MB/s | 457 kB 00:00 2023-01-24T18:54:27.832 INFO:teuthology.orchestra.run.smithi121.stdout:(25/28): python3-markupsafe-0.23-19.el8.x86_64. 1.2 MB/s | 39 kB 00:00 2023-01-24T18:54:27.846 INFO:teuthology.orchestra.run.smithi121.stdout:(26/28): python3-pytz-2017.2-9.el8.noarch.rpm 3.8 MB/s | 54 kB 00:00 2023-01-24T18:54:27.847 INFO:teuthology.orchestra.run.smithi121.stdout:(27/28): python3-mako-1.0.6-13.el8.noarch.rpm 9.8 MB/s | 157 kB 00:00 2023-01-24T18:54:27.911 INFO:teuthology.orchestra.run.smithi121.stdout:(28/28): python3-webob-1.8.5-1.el8.1.noarch.rpm 1.5 MB/s | 251 kB 00:00 2023-01-24T18:54:27.913 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:54:27.914 INFO:teuthology.orchestra.run.smithi121.stdout:Total 20 MB/s | 27 MB 00:01 2023-01-24T18:54:27.918 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:54:27.954 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:54:27.954 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:54:28.346 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:54:28.350 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:54:29.401 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:54:29.640 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/28 2023-01-24T18:54:29.854 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/28 2023-01-24T18:54:30.018 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 3/28 2023-01-24T18:54:30.288 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/28 2023-01-24T18:54:30.648 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/28 2023-01-24T18:54:30.848 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 6/28 2023-01-24T18:54:31.089 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 7/28 2023-01-24T18:54:31.307 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 8/28 2023-01-24T18:54:31.446 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 9/28 2023-01-24T18:54:31.627 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 10/28 2023-01-24T18:54:31.800 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 11/28 2023-01-24T18:54:32.004 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 12/28 2023-01-24T18:54:32.146 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 13/28 2023-01-24T18:54:32.952 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 14/28 2023-01-24T18:54:33.196 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 15/28 2023-01-24T18:54:33.355 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 16/28 2023-01-24T18:54:33.620 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-portend-2.6-1.el8.noarch 17/28 2023-01-24T18:54:33.857 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 18/28 2023-01-24T18:54:34.060 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 19/28 2023-01-24T18:54:34.406 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 20/28 2023-01-24T18:54:34.673 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 21/28 2023-01-24T18:54:34.950 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 22/28 2023-01-24T18:54:35.318 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 23/28 2023-01-24T18:54:35.475 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:54:36.953 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 24/28 2023-01-24T18:54:37.125 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:54:37.517 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 25/28 2023-01-24T18:54:37.725 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:54:37.940 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 26/28 2023-01-24T18:54:38.105 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:54:38.122 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 27/28 2023-01-24T18:54:38.245 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:54:38.640 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 28/28 2023-01-24T18:54:38.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 1/28 2023-01-24T18:54:38.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2/28 2023-01-24T18:54:38.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 3/28 2023-01-24T18:54:38.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 4/28 2023-01-24T18:54:38.641 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 5/28 2023-01-24T18:54:38.642 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.n 6/28 2023-01-24T18:54:38.642 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/28 2023-01-24T18:54:38.642 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/28 2023-01-24T18:54:38.642 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/28 2023-01-24T18:54:38.642 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/28 2023-01-24T18:54:38.643 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/28 2023-01-24T18:54:38.643 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/28 2023-01-24T18:54:38.643 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/28 2023-01-24T18:54:38.643 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/28 2023-01-24T18:54:38.643 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/28 2023-01-24T18:54:38.644 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/28 2023-01-24T18:54:38.644 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/28 2023-01-24T18:54:38.644 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/28 2023-01-24T18:54:38.644 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/28 2023-01-24T18:54:38.644 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/28 2023-01-24T18:54:38.645 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/28 2023-01-24T18:54:38.645 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/28 2023-01-24T18:54:38.645 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/28 2023-01-24T18:54:38.645 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/28 2023-01-24T18:54:38.645 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/28 2023-01-24T18:54:38.646 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/28 2023-01-24T18:54:38.646 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 27/28 2023-01-24T18:54:40.000 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/28 2023-01-24T18:54:40.000 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:25 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:41.862 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:54:41.962 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:41.962 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:54:41.962 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:41.962 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mds-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-modules-core-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mon-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-osd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-24T18:54:41.963 INFO:teuthology.orchestra.run.smithi121.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-24T18:54:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-24T18:54:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-24T18:54:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-24T18:54:41.964 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-24T18:54:41.965 INFO:teuthology.orchestra.run.smithi121.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-24T18:54:41.965 INFO:teuthology.orchestra.run.smithi121.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-24T18:54:41.965 INFO:teuthology.orchestra.run.smithi121.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-24T18:54:41.965 INFO:teuthology.orchestra.run.smithi121.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-24T18:54:41.965 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-24T18:54:41.966 INFO:teuthology.orchestra.run.smithi121.stdout: python3-portend-2.6-1.el8.noarch 2023-01-24T18:54:41.966 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-24T18:54:41.966 INFO:teuthology.orchestra.run.smithi121.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-24T18:54:41.966 INFO:teuthology.orchestra.run.smithi121.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-24T18:54:41.966 INFO:teuthology.orchestra.run.smithi121.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-24T18:54:41.967 INFO:teuthology.orchestra.run.smithi121.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-24T18:54:41.967 INFO:teuthology.orchestra.run.smithi121.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-24T18:54:41.967 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-24T18:54:41.967 INFO:teuthology.orchestra.run.smithi121.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-24T18:54:41.968 INFO:teuthology.orchestra.run.smithi121.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-24T18:54:41.968 INFO:teuthology.orchestra.run.smithi121.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-24T18:54:41.968 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:41.968 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:54:42.117 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-base 2023-01-24T18:54:42.480 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:54:43.226 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:29 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:44.570 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-base-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:54:45.025 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:54:45.026 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:54:45.026 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:54:45.121 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install cephadm 2023-01-24T18:54:45.475 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:54:46.195 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:32 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:47.953 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:54:47.954 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:47.954 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:54:47.954 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:47.955 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:54:47.955 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 78 k 2023-01-24T18:54:47.955 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:47.955 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:54:47.955 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:47.956 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:54:47.956 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:47.956 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 78 k 2023-01-24T18:54:47.956 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 355 k 2023-01-24T18:54:47.956 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:54:48.257 INFO:teuthology.orchestra.run.smithi121.stdout:cephadm-17.2.5-727.g49f8fb05.el8.noarch.rpm 259 kB/s | 78 kB 00:00 2023-01-24T18:54:48.257 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:54:48.258 INFO:teuthology.orchestra.run.smithi121.stdout:Total 258 kB/s | 78 kB 00:00 2023-01-24T18:54:48.258 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:54:48.266 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:54:48.266 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:54:48.267 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:54:48.272 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:54:48.368 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:54:52.514 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:54:52.743 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:54:52.946 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:54:53.860 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:54:53.861 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:39 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:55.714 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:54:55.809 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:55.809 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:54:55.810 INFO:teuthology.orchestra.run.smithi121.stdout: cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:54:55.810 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:55.810 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:54:56.379 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-immutable-object-cache 2023-01-24T18:54:56.747 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:54:57.492 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:43 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:54:59.253 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:54:59.254 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:59.254 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-24T18:54:59.254 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:59.254 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 162 k 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:54:59.255 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:54:59.256 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 162 k 2023-01-24T18:54:59.256 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 490 k 2023-01-24T18:54:59.256 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:54:59.528 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-immutable-object-cache-17.2.5-727.g49f8fb0 592 kB/s | 162 kB 00:00 2023-01-24T18:54:59.529 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:54:59.529 INFO:teuthology.orchestra.run.smithi121.stdout:Total 589 kB/s | 162 kB 00:00 2023-01-24T18:54:59.529 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:54:59.541 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:54:59.541 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:54:59.594 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:54:59.598 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:54:59.721 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:54:59.875 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:55:00.288 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:55:01.225 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.e 1/1 2023-01-24T18:55:01.225 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:47 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:03.175 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:03.272 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:03.273 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:03.273 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-immutable-object-cache-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:55:03.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:03.273 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:03.380 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr 2023-01-24T18:55:03.736 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:04.495 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:50 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:05.807 INFO:teuthology.orchestra.run.smithi121.stdout:Package ceph-mgr-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:55:06.259 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:06.261 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:55:06.261 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:06.356 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-dashboard 2023-01-24T18:55:06.712 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:07.483 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:53 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:09.236 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:09.237 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================================== 2023-01-24T18:55:09.238 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:55:09.238 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================================== 2023-01-24T18:55:09.238 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:55:09.238 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 1.6 M 2023-01-24T18:55:09.238 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:55:09.239 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 23 k 2023-01-24T18:55:09.239 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 14 k 2023-01-24T18:55:09.239 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:55:09.239 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-24T18:55:09.239 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-24T18:55:09.240 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:09.240 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:55:09.240 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================================== 2023-01-24T18:55:09.240 INFO:teuthology.orchestra.run.smithi121.stdout:Install 6 Packages 2023-01-24T18:55:09.240 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:09.241 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 2.0 M 2023-01-24T18:55:09.241 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 7.8 M 2023-01-24T18:55:09.241 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:55:09.525 INFO:teuthology.orchestra.run.smithi121.stdout:(1/6): ceph-prometheus-alerts-17.2.5-727.g49f8f 51 kB/s | 14 kB 00:00 2023-01-24T18:55:09.531 INFO:teuthology.orchestra.run.smithi121.stdout:(2/6): ceph-grafana-dashboards-17.2.5-727.g49f8 79 kB/s | 23 kB 00:00 2023-01-24T18:55:09.696 INFO:teuthology.orchestra.run.smithi121.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 198 kB/s | 34 kB 00:00 2023-01-24T18:55:09.727 INFO:teuthology.orchestra.run.smithi121.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-01-24T18:55:09.806 INFO:teuthology.orchestra.run.smithi121.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 718 kB/s | 196 kB 00:00 2023-01-24T18:55:10.174 INFO:teuthology.orchestra.run.smithi121.stdout:(6/6): ceph-mgr-dashboard-17.2.5-727.g49f8fb05. 1.8 MB/s | 1.6 MB 00:00 2023-01-24T18:55:10.176 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:55:10.176 INFO:teuthology.orchestra.run.smithi121.stdout:Total 2.1 MB/s | 2.0 MB 00:00 2023-01-24T18:55:10.178 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:55:10.186 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:55:10.186 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:55:10.264 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:55:10.268 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:55:10.574 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:55:10.767 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-24T18:55:10.981 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-24T18:55:11.150 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-24T18:55:11.262 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 4/6 2023-01-24T18:55:11.651 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 5/6 2023-01-24T18:55:11.900 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:55:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 6/6 2023-01-24T18:55:12.173 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.n 1/6 2023-01-24T18:55:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2/6 2023-01-24T18:55:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.no 3/6 2023-01-24T18:55:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-24T18:55:12.174 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-24T18:55:13.306 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-24T18:55:13.306 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:01:59 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:15.161 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:15.257 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:15.257 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:15.258 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-grafana-dashboards-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:15.258 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-dashboard-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:15.258 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-prometheus-alerts-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:15.258 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-24T18:55:15.258 INFO:teuthology.orchestra.run.smithi121.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-24T18:55:15.259 INFO:teuthology.orchestra.run.smithi121.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-24T18:55:15.259 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:15.259 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:15.369 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-24T18:55:15.728 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:16.486 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:02 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:18.311 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:18.312 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================================== 2023-01-24T18:55:18.312 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:55:18.312 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================================== 2023-01-24T18:55:18.312 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:55:18.313 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 7.4 M 2023-01-24T18:55:18.313 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:55:18.313 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-01-24T18:55:18.313 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-01-24T18:55:18.313 INFO:teuthology.orchestra.run.smithi121.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-01-24T18:55:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-24T18:55:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-24T18:55:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-24T18:55:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-01-24T18:55:18.314 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:18.315 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:55:18.315 INFO:teuthology.orchestra.run.smithi121.stdout:====================================================================================================================== 2023-01-24T18:55:18.315 INFO:teuthology.orchestra.run.smithi121.stdout:Install 8 Packages 2023-01-24T18:55:18.315 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:18.316 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 36 M 2023-01-24T18:55:18.316 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 199 M 2023-01-24T18:55:18.316 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:55:18.352 INFO:teuthology.orchestra.run.smithi121.stdout:(1/8): libquadmath-8.4.1-1.el8.x86_64.rpm 4.4 MB/s | 169 kB 00:00 2023-01-24T18:55:18.358 INFO:teuthology.orchestra.run.smithi121.stdout:(2/8): libgfortran-8.4.1-1.el8.x86_64.rpm 14 MB/s | 639 kB 00:00 2023-01-24T18:55:18.377 INFO:teuthology.orchestra.run.smithi121.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 11 MB/s | 225 kB 00:00 2023-01-24T18:55:18.441 INFO:teuthology.orchestra.run.smithi121.stdout:(4/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 59 MB/s | 3.7 MB 00:00 2023-01-24T18:55:18.517 INFO:teuthology.orchestra.run.smithi121.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 63 MB/s | 4.7 MB 00:00 2023-01-24T18:55:18.629 INFO:teuthology.orchestra.run.smithi121.stdout:(6/8): python3-scipy-1.0.0-20.module+el8.1.0+33 52 MB/s | 15 MB 00:00 2023-01-24T18:55:18.671 INFO:teuthology.orchestra.run.smithi121.stdout:(7/8): openblas-0.3.12-1.el8.x86_64.rpm 30 MB/s | 4.6 MB 00:00 2023-01-24T18:55:19.364 INFO:teuthology.orchestra.run.smithi121.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.5-727 7.0 MB/s | 7.4 MB 00:01 2023-01-24T18:55:19.365 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:55:19.366 INFO:teuthology.orchestra.run.smithi121.stdout:Total 34 MB/s | 36 MB 00:01 2023-01-24T18:55:19.375 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:55:19.393 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:55:19.394 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:55:19.804 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:55:19.808 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:55:20.402 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:55:20.554 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:55:20.712 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2023-01-24T18:55:20.853 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:55:21.399 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:55:21.570 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:55:22.127 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 3/8 2023-01-24T18:55:22.295 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:55:23.027 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 4/8 2023-01-24T18:55:23.430 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:55:23.669 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:55:25.713 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:55:26.953 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 7/8 2023-01-24T18:55:27.144 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:55:27.436 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 8/8 2023-01-24T18:55:27.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05 1/8 2023-01-24T18:55:27.436 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 2/8 2023-01-24T18:55:27.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 3/8 2023-01-24T18:55:27.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7. 4/8 2023-01-24T18:55:27.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2023-01-24T18:55:27.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 6/8 2023-01-24T18:55:27.437 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 7/8 2023-01-24T18:55:28.369 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 8/8 2023-01-24T18:55:28.370 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:14 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:30.277 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:30.373 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:30.374 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:30.374 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-diskprediction-local-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:30.375 INFO:teuthology.orchestra.run.smithi121.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-01-24T18:55:30.375 INFO:teuthology.orchestra.run.smithi121.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-01-24T18:55:30.375 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-0.3.12-1.el8.x86_64 2023-01-24T18:55:30.375 INFO:teuthology.orchestra.run.smithi121.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-01-24T18:55:30.376 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-01-24T18:55:30.376 INFO:teuthology.orchestra.run.smithi121.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-01-24T18:55:30.376 INFO:teuthology.orchestra.run.smithi121.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-01-24T18:55:30.376 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:30.377 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:30.546 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-rook 2023-01-24T18:55:30.908 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:31.670 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:17 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:33.445 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:33.447 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-24T18:55:33.447 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:55:33.447 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-24T18:55:33.447 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:55:33.448 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 47 k 2023-01-24T18:55:33.448 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:55:33.448 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-24T18:55:33.448 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-24T18:55:33.449 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-24T18:55:33.449 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-24T18:55:33.449 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-24T18:55:33.449 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-24T18:55:33.449 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-24T18:55:33.450 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:33.451 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:55:33.451 INFO:teuthology.orchestra.run.smithi121.stdout:======================================================================================================= 2023-01-24T18:55:33.451 INFO:teuthology.orchestra.run.smithi121.stdout:Install 13 Packages 2023-01-24T18:55:33.451 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:33.452 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 1.9 M 2023-01-24T18:55:33.452 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 25 M 2023-01-24T18:55:33.452 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:55:33.639 INFO:teuthology.orchestra.run.smithi121.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 86 kB/s | 16 kB 00:00 2023-01-24T18:55:33.948 INFO:teuthology.orchestra.run.smithi121.stdout:(2/13): ceph-mgr-rook-17.2.5-727.g49f8fb05.el8. 94 kB/s | 47 kB 00:00 2023-01-24T18:55:33.998 INFO:teuthology.orchestra.run.smithi121.stdout:(3/13): python3-cachetools-3.1.1-4.el8.noarch.r 60 kB/s | 33 kB 00:00 2023-01-24T18:55:34.150 INFO:teuthology.orchestra.run.smithi121.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 159 kB/s | 81 kB 00:00 2023-01-24T18:55:34.183 INFO:teuthology.orchestra.run.smithi121.stdout:(5/13): python3-rsa-4.8-1.el8.noarch.rpm 333 kB/s | 61 kB 00:00 2023-01-24T18:55:34.216 INFO:teuthology.orchestra.run.smithi121.stdout:(6/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 4.6 MB/s | 155 kB 00:00 2023-01-24T18:55:34.230 INFO:teuthology.orchestra.run.smithi121.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 3.1 MB/s | 43 kB 00:00 2023-01-24T18:55:34.243 INFO:teuthology.orchestra.run.smithi121.stdout:(8/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 2.0 MB/s | 27 kB 00:00 2023-01-24T18:55:34.256 INFO:teuthology.orchestra.run.smithi121.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 9.1 MB/s | 126 kB 00:00 2023-01-24T18:55:34.271 INFO:teuthology.orchestra.run.smithi121.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 7.4 MB/s | 110 kB 00:00 2023-01-24T18:55:34.284 INFO:teuthology.orchestra.run.smithi121.stdout:(11/13): python3-jsonpointer-1.10-11.el8.noarch 1.6 MB/s | 20 kB 00:00 2023-01-24T18:55:34.298 INFO:teuthology.orchestra.run.smithi121.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 414 kB/s | 61 kB 00:00 2023-01-24T18:55:34.948 INFO:teuthology.orchestra.run.smithi121.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 1.2 MB/s | 1.2 MB 00:00 2023-01-24T18:55:34.949 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:55:34.949 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.3 MB/s | 1.9 MB 00:01 2023-01-24T18:55:34.956 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:55:34.971 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:55:34.971 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:55:35.071 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:55:35.076 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:55:35.591 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:55:35.872 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-24T18:55:36.107 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-24T18:55:36.290 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-24T18:55:36.533 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-24T18:55:36.776 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-24T18:55:37.059 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-24T18:55:37.307 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-24T18:55:37.559 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-24T18:55:37.761 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-24T18:55:38.040 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-24T18:55:39.191 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-24T18:55:39.477 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-24T18:55:39.599 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:55:39.868 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 13/13 2023-01-24T18:55:39.868 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 1/13 2023-01-24T18:55:39.869 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-24T18:55:39.869 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-24T18:55:39.869 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-24T18:55:39.869 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-24T18:55:39.869 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2023-01-24T18:55:39.870 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 12/13 2023-01-24T18:55:40.884 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-24T18:55:40.885 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:26 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:42.764 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:42.866 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:42.867 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:42.867 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-rook-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:42.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-24T18:55:42.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-24T18:55:42.867 INFO:teuthology.orchestra.run.smithi121.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-24T18:55:42.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-24T18:55:42.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-24T18:55:42.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-24T18:55:42.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-24T18:55:42.868 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-24T18:55:42.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-24T18:55:42.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-24T18:55:42.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-24T18:55:42.869 INFO:teuthology.orchestra.run.smithi121.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-24T18:55:42.869 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:42.870 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:43.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-mgr-cephadm 2023-01-24T18:55:43.366 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:44.134 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:30 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:45.927 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:45.928 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================== 2023-01-24T18:55:45.928 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:55:45.928 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================== 2023-01-24T18:55:45.928 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:55:45.928 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 115 k 2023-01-24T18:55:45.929 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:55:45.929 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-24T18:55:45.929 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-24T18:55:45.929 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-24T18:55:45.929 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-24T18:55:45.930 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:45.930 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:55:45.930 INFO:teuthology.orchestra.run.smithi121.stdout:============================================================================================== 2023-01-24T18:55:45.930 INFO:teuthology.orchestra.run.smithi121.stdout:Install 5 Packages 2023-01-24T18:55:45.930 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:45.931 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 5.9 M 2023-01-24T18:55:45.931 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 26 M 2023-01-24T18:55:45.931 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:55:46.050 INFO:teuthology.orchestra.run.smithi121.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 496 kB/s | 60 kB 00:00 2023-01-24T18:55:46.111 INFO:teuthology.orchestra.run.smithi121.stdout:(2/5): ceph-mgr-cephadm-17.2.5-727.g49f8fb05.el 630 kB/s | 115 kB 00:00 2023-01-24T18:55:46.130 INFO:teuthology.orchestra.run.smithi121.stdout:(3/5): python3-babel-2.5.1-5.el8.noarch.rpm 60 MB/s | 4.8 MB 00:00 2023-01-24T18:55:46.153 INFO:teuthology.orchestra.run.smithi121.stdout:(4/5): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 13 MB/s | 538 kB 00:00 2023-01-24T18:55:46.160 INFO:teuthology.orchestra.run.smithi121.stdout:(5/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 2.0 MB/s | 476 kB 00:00 2023-01-24T18:55:46.161 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:55:46.161 INFO:teuthology.orchestra.run.smithi121.stdout:Total 25 MB/s | 5.9 MB 00:00 2023-01-24T18:55:46.164 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:55:46.174 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:55:46.175 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:55:46.267 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:55:46.271 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:55:47.271 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:55:47.621 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 1/5 2023-01-24T18:55:47.859 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 2/5 2023-01-24T18:55:48.188 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:55:48.443 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-24T18:55:48.639 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:55:48.894 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 5/5 2023-01-24T18:55:48.895 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 1/5 2023-01-24T18:55:48.895 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-24T18:55:48.895 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-24T18:55:48.895 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 4/5 2023-01-24T18:55:49.775 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 5/5 2023-01-24T18:55:49.775 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:35 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:51.584 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:51.683 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:51.684 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:51.684 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-mgr-cephadm-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:55:51.684 INFO:teuthology.orchestra.run.smithi121.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-24T18:55:51.684 INFO:teuthology.orchestra.run.smithi121.stdout: python3-babel-2.5.1-5.el8.noarch 2023-01-24T18:55:51.684 INFO:teuthology.orchestra.run.smithi121.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-01-24T18:55:51.685 INFO:teuthology.orchestra.run.smithi121.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-24T18:55:51.685 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:51.685 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:51.796 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-fuse 2023-01-24T18:55:52.161 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:55:52.893 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:38 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:55.156 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:55:55.157 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:55:55.157 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:55:55.158 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:55:55.158 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:55:55.158 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse 2023-01-24T18:55:55.158 INFO:teuthology.orchestra.run.smithi121.stdout: x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 872 k 2023-01-24T18:55:55.158 INFO:teuthology.orchestra.run.smithi121.stdout:Installing dependencies: 2023-01-24T18:55:55.159 INFO:teuthology.orchestra.run.smithi121.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-24T18:55:55.159 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:55.159 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:55:55.159 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:55:55.160 INFO:teuthology.orchestra.run.smithi121.stdout:Install 2 Packages 2023-01-24T18:55:55.160 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:55.160 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 954 k 2023-01-24T18:55:55.160 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 3.0 M 2023-01-24T18:55:55.160 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:55:55.185 INFO:teuthology.orchestra.run.smithi121.stdout:(1/2): fuse-2.9.7-12.el8.x86_64.rpm 3.0 MB/s | 83 kB 00:00 2023-01-24T18:55:55.399 INFO:teuthology.orchestra.run.smithi121.stdout:(2/2): ceph-fuse-17.2.5-727.g49f8fb05.el8.x86_6 3.5 MB/s | 872 kB 00:00 2023-01-24T18:55:55.400 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:55:55.400 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.8 MB/s | 954 kB 00:00 2023-01-24T18:55:55.402 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:55:55.422 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:55:55.423 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:55:55.503 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:55:55.507 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:55:55.702 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:55:55.947 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2023-01-24T18:55:56.112 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:55:56.479 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2/2 2023-01-24T18:55:56.480 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/2 2023-01-24T18:55:57.400 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2023-01-24T18:55:57.400 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:43 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:55:59.336 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:55:59.434 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:59.434 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:55:59.434 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2023-01-24T18:55:59.434 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:55:59.435 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:55:59.554 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install ceph-volume 2023-01-24T18:55:59.923 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:00.700 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:46 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:02.459 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:02.460 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:02.461 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repository Size 2023-01-24T18:56:02.461 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:02.461 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:02.461 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume noarch 2:17.2.5-727.g49f8fb05.el8 ceph-noarch 263 k 2023-01-24T18:56:02.461 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:02.462 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:02.462 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:02.462 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:02.462 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:02.462 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 263 k 2023-01-24T18:56:02.463 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 1.3 M 2023-01-24T18:56:02.463 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:02.691 INFO:teuthology.orchestra.run.smithi121.stdout:ceph-volume-17.2.5-727.g49f8fb05.el8.noarch.rpm 1.1 MB/s | 263 kB 00:00 2023-01-24T18:56:02.692 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:02.693 INFO:teuthology.orchestra.run.smithi121.stdout:Total 1.1 MB/s | 263 kB 00:00 2023-01-24T18:56:02.693 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:02.703 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:02.703 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:02.730 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:02.734 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:02.917 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:03.203 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:56:03.608 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:56:04.546 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 1/1 2023-01-24T18:56:04.546 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:50 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:06.457 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:56:06.554 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:06.554 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:56:06.554 INFO:teuthology.orchestra.run.smithi121.stdout: ceph-volume-2:17.2.5-727.g49f8fb05.el8.noarch 2023-01-24T18:56:06.554 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:06.555 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:06.670 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados-devel 2023-01-24T18:56:07.035 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:07.885 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:53 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:09.645 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:09.646 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:09.646 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-24T18:56:09.646 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:09.646 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:09.646 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 130 k 2023-01-24T18:56:09.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:09.647 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:09.647 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:09.647 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:09.647 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:09.648 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 130 k 2023-01-24T18:56:09.648 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 464 k 2023-01-24T18:56:09.648 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:09.814 INFO:teuthology.orchestra.run.smithi121.stdout:librados-devel-17.2.5-727.g49f8fb05.el8.x86_64. 777 kB/s | 130 kB 00:00 2023-01-24T18:56:09.815 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:09.815 INFO:teuthology.orchestra.run.smithi121.stdout:Total 770 kB/s | 130 kB 00:00 2023-01-24T18:56:09.815 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:09.821 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:09.821 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:09.877 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:09.881 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:10.010 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:10.183 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:10.448 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:11.340 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:11.340 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:02:57 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:13.185 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:56:13.279 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:13.280 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:56:13.280 INFO:teuthology.orchestra.run.smithi121.stdout: librados-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:56:13.280 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:13.280 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:13.411 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs2 2023-01-24T18:56:13.779 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:14.558 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:00 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:15.884 INFO:teuthology.orchestra.run.smithi121.stdout:Package libcephfs2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:16.344 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:16.345 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:16.345 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:16.443 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install libcephfs-devel 2023-01-24T18:56:16.811 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:17.545 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:03 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:19.321 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:19.322 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:19.322 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-24T18:56:19.322 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:19.322 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:19.323 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 25 k 2023-01-24T18:56:19.323 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:19.323 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:19.323 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:19.323 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:19.324 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:19.324 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 25 k 2023-01-24T18:56:19.324 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 109 k 2023-01-24T18:56:19.324 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:19.501 INFO:teuthology.orchestra.run.smithi121.stdout:libcephfs-devel-17.2.5-727.g49f8fb05.el8.x86_64 139 kB/s | 25 kB 00:00 2023-01-24T18:56:19.501 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:19.502 INFO:teuthology.orchestra.run.smithi121.stdout:Total 138 kB/s | 25 kB 00:00 2023-01-24T18:56:19.502 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:19.506 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:19.506 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:19.507 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:19.512 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:19.562 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:19.749 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:19.985 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:20.863 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:20.863 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:06 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:22.789 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:56:22.886 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:22.886 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:56:22.887 INFO:teuthology.orchestra.run.smithi121.stdout: libcephfs-devel-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:56:22.887 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:22.887 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:23.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librados2 2023-01-24T18:56:23.375 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:24.141 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:10 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:25.515 INFO:teuthology.orchestra.run.smithi121.stdout:Package librados2-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:25.962 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:25.963 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:25.964 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:26.062 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install librbd1 2023-01-24T18:56:26.426 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:27.162 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:13 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:28.481 INFO:teuthology.orchestra.run.smithi121.stdout:Package librbd1-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:28.934 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:28.935 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:28.935 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:29.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rados 2023-01-24T18:56:29.392 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:30.145 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:16 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:31.447 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rados-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:31.907 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:31.909 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:31.909 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:32.006 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rgw 2023-01-24T18:56:32.372 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:33.146 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:19 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:34.488 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rgw-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:34.938 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:34.939 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:34.939 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:35.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-cephfs 2023-01-24T18:56:35.406 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:36.126 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:22 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:37.504 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-cephfs-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:37.962 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:37.963 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:37.963 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:38.068 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install python3-rbd 2023-01-24T18:56:38.434 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:39.184 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:25 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:40.470 INFO:teuthology.orchestra.run.smithi121.stdout:Package python3-rbd-2:17.2.5-727.g49f8fb05.el8.x86_64 is already installed. 2023-01-24T18:56:40.920 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:40.921 INFO:teuthology.orchestra.run.smithi121.stdout:Nothing to do. 2023-01-24T18:56:40.922 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:41.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-fuse 2023-01-24T18:56:41.375 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:42.235 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:28 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:43.992 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:43.993 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:43.993 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-01-24T18:56:43.994 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:43.994 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:43.994 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 90 k 2023-01-24T18:56:43.994 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:43.994 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:43.995 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:43.995 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:43.995 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:43.995 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 90 k 2023-01-24T18:56:43.995 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 264 k 2023-01-24T18:56:43.996 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:44.166 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-fuse-17.2.5-727.g49f8fb05.el8.x86_64.rpm 524 kB/s | 90 kB 00:00 2023-01-24T18:56:44.167 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:44.167 INFO:teuthology.orchestra.run.smithi121.stdout:Total 520 kB/s | 90 kB 00:00 2023-01-24T18:56:44.168 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:44.178 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:44.179 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:44.233 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:44.237 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:44.348 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:44.551 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:44.797 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:45.701 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:45.701 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:31 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:47.549 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:56:47.644 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:47.645 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:56:47.645 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-fuse-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:56:47.645 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:47.645 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:47.757 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-mirror 2023-01-24T18:56:48.122 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:48.873 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:34 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:50.632 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:50.633 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:50.633 INFO:teuthology.orchestra.run.smithi121.stdout: Package Arch Version Repo Size 2023-01-24T18:56:50.633 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 3.5 M 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:50.634 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:50.635 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:50.635 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 3.5 M 2023-01-24T18:56:50.635 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 16 M 2023-01-24T18:56:50.635 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:51.544 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-mirror-17.2.5-727.g49f8fb05.el8.x86_64.rpm 3.8 MB/s | 3.5 MB 00:00 2023-01-24T18:56:51.545 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:51.545 INFO:teuthology.orchestra.run.smithi121.stdout:Total 3.8 MB/s | 3.5 MB 00:00 2023-01-24T18:56:51.546 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:51.560 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:51.560 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:51.628 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:51.631 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:52.069 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:52.259 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:52.653 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:53.585 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:53.585 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:39 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:55.413 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:56:55.508 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:55.508 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:56:55.509 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-mirror-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:56:55.509 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:55.509 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:56:55.629 DEBUG:teuthology.orchestra.run.smithi121:> sudo yum -y install rbd-nbd 2023-01-24T18:56:55.997 INFO:teuthology.orchestra.run.smithi121.stdout:Updating Subscription Management repositories. 2023-01-24T18:56:56.728 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:42 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:56:58.494 INFO:teuthology.orchestra.run.smithi121.stdout:Dependencies resolved. 2023-01-24T18:56:58.495 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:58.495 INFO:teuthology.orchestra.run.smithi121.stdout: Package Architecture Version Repository Size 2023-01-24T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:Installing: 2023-01-24T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd x86_64 2:17.2.5-727.g49f8fb05.el8 ceph 172 k 2023-01-24T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:58.496 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction Summary 2023-01-24T18:56:58.497 INFO:teuthology.orchestra.run.smithi121.stdout:================================================================================ 2023-01-24T18:56:58.497 INFO:teuthology.orchestra.run.smithi121.stdout:Install 1 Package 2023-01-24T18:56:58.497 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:56:58.497 INFO:teuthology.orchestra.run.smithi121.stdout:Total download size: 172 k 2023-01-24T18:56:58.497 INFO:teuthology.orchestra.run.smithi121.stdout:Installed size: 548 k 2023-01-24T18:56:58.498 INFO:teuthology.orchestra.run.smithi121.stdout:Downloading Packages: 2023-01-24T18:56:58.675 INFO:teuthology.orchestra.run.smithi121.stdout:rbd-nbd-17.2.5-727.g49f8fb05.el8.x86_64.rpm 960 kB/s | 172 kB 00:00 2023-01-24T18:56:58.676 INFO:teuthology.orchestra.run.smithi121.stdout:-------------------------------------------------------------------------------- 2023-01-24T18:56:58.677 INFO:teuthology.orchestra.run.smithi121.stdout:Total 952 kB/s | 172 kB 00:00 2023-01-24T18:56:58.677 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction check 2023-01-24T18:56:58.689 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction check succeeded. 2023-01-24T18:56:58.689 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction test 2023-01-24T18:56:58.746 INFO:teuthology.orchestra.run.smithi121.stdout:Transaction test succeeded. 2023-01-24T18:56:58.750 INFO:teuthology.orchestra.run.smithi121.stdout:Running transaction 2023-01-24T18:56:58.880 INFO:teuthology.orchestra.run.smithi121.stdout: Preparing : 1/1 2023-01-24T18:56:59.048 INFO:teuthology.orchestra.run.smithi121.stdout: Installing : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:56:59.312 INFO:teuthology.orchestra.run.smithi121.stdout: Running scriptlet: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:57:00.205 INFO:teuthology.orchestra.run.smithi121.stdout: Verifying : rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 1/1 2023-01-24T18:57:00.206 INFO:teuthology.orchestra.run.smithi121.stdout:Last metadata expiration check: 0:03:46 ago on Tue 24 Jan 2023 06:53:14 PM UTC. 2023-01-24T18:57:02.075 INFO:teuthology.orchestra.run.smithi121.stdout:Installed products updated. 2023-01-24T18:57:02.170 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:02.170 INFO:teuthology.orchestra.run.smithi121.stdout:Installed: 2023-01-24T18:57:02.170 INFO:teuthology.orchestra.run.smithi121.stdout: rbd-nbd-2:17.2.5-727.g49f8fb05.el8.x86_64 2023-01-24T18:57:02.171 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:02.171 INFO:teuthology.orchestra.run.smithi121.stdout:Complete! 2023-01-24T18:57:02.280 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:02.280 DEBUG:teuthology.orchestra.run.smithi121:> sudo lsb_release -is 2023-01-24T18:57:02.317 INFO:teuthology.orchestra.run.smithi121.stdout:RedHatEnterprise 2023-01-24T18:57:02.319 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-24T18:57:02.319 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:57:02.525 DEBUG:teuthology.orchestra.run.smithi121:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-24T18:57:02.544 INFO:teuthology.orchestra.run.smithi121.stdout:17.2.5-727.g49f8fb05.el8 2023-01-24T18:57:02.545 INFO:teuthology.packaging:The installed version of ceph is 17.2.5-727.g49f8fb05.el8 2023-01-24T18:57:02.545 INFO:teuthology.task.install:The correct ceph version 17.2.5-727.g49f8fb05 is installed. 2023-01-24T18:57:02.546 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-24T18:57:02.546 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:02.547 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-24T18:57:02.608 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-24T18:57:02.609 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:02.609 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/daemon-helper 2023-01-24T18:57:02.671 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-24T18:57:02.734 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-24T18:57:02.735 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:02.735 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-24T18:57:02.797 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-24T18:57:02.860 INFO:teuthology.run_tasks:Running task ceph... 2023-01-24T18:57:02.927 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-24T18:57:02.928 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 777 /var/log/ceph 2023-01-24T18:57:02.952 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-24T18:57:02.952 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-24T18:57:03.016 INFO:tasks.ceph:Creating extra log directories... 2023-01-24T18:57:03.016 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-24T18:57:03.087 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-24T18:57:03.087 INFO:tasks.ceph:config {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-24T18:57:03.088 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi/7135617', 'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'description': 'rados/singleton-nomsgr/{all/health-warnings mon_election/connectivity rados supported-random-distro$/{rhel_8}}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7135617', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_16:05:13-rados-wip-yuri7-testing-2023-01-23-1532-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0']], 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri7-testing-2023-01-23-1532-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_49f8fb05584886826e8eade75f7105fba754560c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'targets': {'smithi121.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGJNlc8wQwRp5m7j/w2nuuTbMzSNL1oHGr/aqbttTjsH3t0nC5ZaKr7kp9YVzV0OCBoOzqXm+JLDX7DI/ykEzyc='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 3}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': '49f8fb05584886826e8eade75f7105fba754560c', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rados/test_health_warnings.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_16:05:13', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-24T18:57:03.089 DEBUG:teuthology.orchestra.run.smithi121:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-24T18:57:03.133 DEBUG:teuthology.orchestra.run.smithi121:> sudo install -d -m0777 -- /var/run/ceph 2023-01-24T18:57:03.195 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:03.196 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/scratch_devs of=/dev/stdout 2023-01-24T18:57:03.250 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-24T18:57:03.250 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_1 2023-01-24T18:57:03.303 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-24T18:57:03.304 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:57:03.304 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 202276 Links: 1 2023-01-24T18:57:03.304 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:57:03.304 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:57:03.305 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-24 18:56:52.429409125 +0000 2023-01-24T18:57:03.305 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-24 18:52:33.965398561 +0000 2023-01-24T18:57:03.305 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-24 18:52:33.965398561 +0000 2023-01-24T18:57:03.305 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-24T18:57:03.305 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-24T18:57:03.365 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-24T18:57:03.366 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-24T18:57:03.366 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.00026078 s, 2.0 MB/s 2023-01-24T18:57:03.367 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-24T18:57:03.422 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_2 2023-01-24T18:57:03.477 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-24T18:57:03.478 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:57:03.478 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 204086 Links: 1 2023-01-24T18:57:03.478 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:57:03.478 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:57:03.479 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-24 18:56:52.429409125 +0000 2023-01-24T18:57:03.479 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-24 18:52:34.460391724 +0000 2023-01-24T18:57:03.479 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-24 18:52:34.460391724 +0000 2023-01-24T18:57:03.479 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-24T18:57:03.480 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-24T18:57:03.539 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-24T18:57:03.539 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-24T18:57:03.540 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000217942 s, 2.3 MB/s 2023-01-24T18:57:03.541 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-24T18:57:03.595 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_3 2023-01-24T18:57:03.649 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-24T18:57:03.650 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:57:03.650 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 203597 Links: 1 2023-01-24T18:57:03.650 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:57:03.650 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:57:03.651 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-24 18:56:52.429409125 +0000 2023-01-24T18:57:03.651 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-24 18:52:34.970384680 +0000 2023-01-24T18:57:03.651 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-24 18:52:34.970384680 +0000 2023-01-24T18:57:03.651 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-24T18:57:03.651 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-24T18:57:03.711 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-24T18:57:03.712 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-24T18:57:03.712 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000215692 s, 2.4 MB/s 2023-01-24T18:57:03.713 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-24T18:57:03.768 DEBUG:teuthology.orchestra.run.smithi121:> stat /dev/vg_nvme/lv_4 2023-01-24T18:57:03.823 INFO:teuthology.orchestra.run.smithi121.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-24T18:57:03.823 INFO:teuthology.orchestra.run.smithi121.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-24T18:57:03.823 INFO:teuthology.orchestra.run.smithi121.stdout:Device: 6h/6d Inode: 200333 Links: 1 2023-01-24T18:57:03.824 INFO:teuthology.orchestra.run.smithi121.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-24T18:57:03.824 INFO:teuthology.orchestra.run.smithi121.stdout:Context: system_u:object_r:device_t:s0 2023-01-24T18:57:03.824 INFO:teuthology.orchestra.run.smithi121.stdout:Access: 2023-01-24 18:56:52.429409125 +0000 2023-01-24T18:57:03.824 INFO:teuthology.orchestra.run.smithi121.stdout:Modify: 2023-01-24 18:52:35.619375724 +0000 2023-01-24T18:57:03.824 INFO:teuthology.orchestra.run.smithi121.stdout:Change: 2023-01-24 18:52:35.619375724 +0000 2023-01-24T18:57:03.825 INFO:teuthology.orchestra.run.smithi121.stdout: Birth: - 2023-01-24T18:57:03.825 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-24T18:57:03.885 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records in 2023-01-24T18:57:03.885 INFO:teuthology.orchestra.run.smithi121.stderr:1+0 records out 2023-01-24T18:57:03.886 INFO:teuthology.orchestra.run.smithi121.stderr:512 bytes copied, 0.000218444 s, 2.3 MB/s 2023-01-24T18:57:03.887 DEBUG:teuthology.orchestra.run.smithi121:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-24T18:57:03.941 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:03.942 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:57:03.942 INFO:tasks.ceph:Generating config... 2023-01-24T18:57:03.943 INFO:tasks.ceph:[osd] osd max object name len = 400 2023-01-24T18:57:03.944 INFO:tasks.ceph:[osd] osd max object namespace len = 64 2023-01-24T18:57:03.944 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-24T18:57:03.944 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-24T18:57:03.945 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-24T18:57:03.945 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-24T18:57:03.945 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-24T18:57:03.946 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-24T18:57:03.946 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-24T18:57:03.946 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-24T18:57:03.946 INFO:tasks.ceph:[global] mon election default strategy = 3 2023-01-24T18:57:03.947 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-24T18:57:03.947 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-24T18:57:03.947 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-24T18:57:03.948 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-24T18:57:03.948 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-24T18:57:03.948 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-24T18:57:03.948 INFO:tasks.ceph:Setting up mon.a... 2023-01-24T18:57:03.949 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-24T18:57:04.027 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.keyring 2023-01-24T18:57:04.030 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-24T18:57:04.068 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:57:04.131 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.121')] 2023-01-24T18:57:04.132 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.121', 'mon election default strategy': 3}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2023-01-24T18:57:04.133 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:04.133 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:57:04.185 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.121 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:04.259 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:04.260 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: generated fsid b27849be-aeb9-4bd2-adcf-d31dbae5a00d 2023-01-24T18:57:04.260 INFO:teuthology.orchestra.run.smithi121.stdout:setting min_mon_release = octopus 2023-01-24T18:57:04.260 INFO:teuthology.orchestra.run.smithi121.stdout:epoch 0 2023-01-24T18:57:04.261 INFO:teuthology.orchestra.run.smithi121.stdout:fsid b27849be-aeb9-4bd2-adcf-d31dbae5a00d 2023-01-24T18:57:04.261 INFO:teuthology.orchestra.run.smithi121.stdout:last_changed 2023-01-24T18:57:04.259254+0000 2023-01-24T18:57:04.261 INFO:teuthology.orchestra.run.smithi121.stdout:created 2023-01-24T18:57:04.259254+0000 2023-01-24T18:57:04.261 INFO:teuthology.orchestra.run.smithi121.stdout:min_mon_release 15 (octopus) 2023-01-24T18:57:04.261 INFO:teuthology.orchestra.run.smithi121.stdout:election_strategy: 3 2023-01-24T18:57:04.262 INFO:teuthology.orchestra.run.smithi121.stdout:0: [v2:172.21.15.121:3300/0,v1:172.21.15.121:6789/0] mon.a 2023-01-24T18:57:04.262 INFO:teuthology.orchestra.run.smithi121.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-01-24T18:57:04.263 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-24T18:57:04.316 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b27849be-aeb9-4bd2-adcf-d31dbae5a00d... 2023-01-24T18:57:04.317 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-24T18:57:04.397 INFO:teuthology.orchestra.run.smithi121.stdout:[global] 2023-01-24T18:57:04.397 INFO:teuthology.orchestra.run.smithi121.stdout: chdir = "" 2023-01-24T18:57:04.397 INFO:teuthology.orchestra.run.smithi121.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-24T18:57:04.397 INFO:teuthology.orchestra.run.smithi121.stdout: auth supported = cephx 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: filestore xattr use omap = true 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: mon clock drift allowed = 1.000 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.398 INFO:teuthology.orchestra.run.smithi121.stdout: osd crush chooseleaf type = 0 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: auth debug = true 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on old message = true 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: ms die on bug = true 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.399 INFO:teuthology.orchestra.run.smithi121.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-24T18:57:04.400 INFO:teuthology.orchestra.run.smithi121.stdout: mon pg warn max object skew = 0 2023-01-24T18:57:04.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.400 INFO:teuthology.orchestra.run.smithi121.stdout: # disable pg_autoscaler by default for new pools 2023-01-24T18:57:04.400 INFO:teuthology.orchestra.run.smithi121.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-24T18:57:04.400 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default size = 2 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow primary affinity = true 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd allow pg remap = true 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on legacy crush tunables = false 2023-01-24T18:57:04.401 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on crush straw calc version zero = false 2023-01-24T18:57:04.402 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on no sortbitwise = false 2023-01-24T18:57:04.402 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on osd down out interval zero = false 2023-01-24T18:57:04.402 INFO:teuthology.orchestra.run.smithi121.stdout: mon warn on too few osds = false 2023-01-24T18:57:04.402 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-24T18:57:04.402 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: mon_allow_pool_size_one = true 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: osd default data pool replay window = 5 2023-01-24T18:57:04.403 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.404 INFO:teuthology.orchestra.run.smithi121.stdout: mon allow pool delete = true 2023-01-24T18:57:04.404 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.404 INFO:teuthology.orchestra.run.smithi121.stdout: mon cluster log file level = debug 2023-01-24T18:57:04.404 INFO:teuthology.orchestra.run.smithi121.stdout: debug asserts on shutdown = true 2023-01-24T18:57:04.404 INFO:teuthology.orchestra.run.smithi121.stdout: mon health detail to clog = false 2023-01-24T18:57:04.405 INFO:teuthology.orchestra.run.smithi121.stdout: mon host = 172.21.15.121 2023-01-24T18:57:04.405 INFO:teuthology.orchestra.run.smithi121.stdout: mon election default strategy = 3 2023-01-24T18:57:04.405 INFO:teuthology.orchestra.run.smithi121.stdout: fsid = b27849be-aeb9-4bd2-adcf-d31dbae5a00d 2023-01-24T18:57:04.405 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.405 INFO:teuthology.orchestra.run.smithi121.stdout:[osd] 2023-01-24T18:57:04.406 INFO:teuthology.orchestra.run.smithi121.stdout: osd journal size = 100 2023-01-24T18:57:04.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.406 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub load threshold = 5.0 2023-01-24T18:57:04.406 INFO:teuthology.orchestra.run.smithi121.stdout: osd scrub max interval = 600 2023-01-24T18:57:04.406 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: osd recover clone overlap = true 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: osd recovery max chunk = 1048576 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug shutdown = true 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug op order = true 2023-01-24T18:57:04.407 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify stray on activate = true 2023-01-24T18:57:04.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.408 INFO:teuthology.orchestra.run.smithi121.stdout: osd open classes on start = true 2023-01-24T18:57:04.408 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug pg log writeout = true 2023-01-24T18:57:04.408 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.408 INFO:teuthology.orchestra.run.smithi121.stdout: osd deep scrub update digest min age = 30 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: osd map max advance = 10 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: journal zero on create = true 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.409 INFO:teuthology.orchestra.run.smithi121.stdout: filestore ondisk finisher threads = 3 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: filestore apply finisher threads = 3 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: bdev debug aio = true 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug misdirected ops = true 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: osd max object name len = 400 2023-01-24T18:57:04.410 INFO:teuthology.orchestra.run.smithi121.stdout: osd max object namespace len = 64 2023-01-24T18:57:04.411 INFO:teuthology.orchestra.run.smithi121.stdout: bluestore zero block detection = True 2023-01-24T18:57:04.411 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-24T18:57:04.411 INFO:teuthology.orchestra.run.smithi121.stdout: debug osd = 20 2023-01-24T18:57:04.411 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify cached snaps = True 2023-01-24T18:57:04.411 INFO:teuthology.orchestra.run.smithi121.stdout: osd debug verify missing on start = True 2023-01-24T18:57:04.412 INFO:teuthology.orchestra.run.smithi121.stdout: osd mclock override recovery settings = True 2023-01-24T18:57:04.412 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue = debug_random 2023-01-24T18:57:04.412 INFO:teuthology.orchestra.run.smithi121.stdout: osd op queue cut off = debug_random 2023-01-24T18:57:04.412 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.412 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr] 2023-01-24T18:57:04.413 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-24T18:57:04.413 INFO:teuthology.orchestra.run.smithi121.stdout: debug mgr = 20 2023-01-24T18:57:04.413 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-01-24T18:57:04.413 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-01-24T18:57:04.413 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout: mgr/telemetry/nag = false 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout:[mon] 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout: debug ms = 1 2023-01-24T18:57:04.414 INFO:teuthology.orchestra.run.smithi121.stdout: debug mon = 20 2023-01-24T18:57:04.415 INFO:teuthology.orchestra.run.smithi121.stdout: debug paxos = 20 2023-01-24T18:57:04.415 INFO:teuthology.orchestra.run.smithi121.stdout: debug auth = 20 2023-01-24T18:57:04.415 INFO:teuthology.orchestra.run.smithi121.stdout: mon data avail warn = 5 2023-01-24T18:57:04.415 INFO:teuthology.orchestra.run.smithi121.stdout: mon mgr mkfs grace = 240 2023-01-24T18:57:04.415 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min pgs per osd = 4 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd reporter subtree level = osd 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: mon osd prime pg temp = true 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: mon reweight min bytes per osd = 10 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-24T18:57:04.416 INFO:teuthology.orchestra.run.smithi121.stdout: auth mon ticket ttl = 660 # 11m 2023-01-24T18:57:04.417 INFO:teuthology.orchestra.run.smithi121.stdout: auth service ticket ttl = 240 # 4m 2023-01-24T18:57:04.417 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.417 INFO:teuthology.orchestra.run.smithi121.stdout: # don't complain about insecure global_id in the test suite 2023-01-24T18:57:04.417 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-24T18:57:04.417 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-24T18:57:04.418 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.418 INFO:teuthology.orchestra.run.smithi121.stdout: # 1m isn't quite enough 2023-01-24T18:57:04.418 INFO:teuthology.orchestra.run.smithi121.stdout: mon_down_mkfs_grace = 2m 2023-01-24T18:57:04.418 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.418 INFO:teuthology.orchestra.run.smithi121.stdout: mon_warn_on_filestore_osds = false 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: mon scrub interval = 300 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout:[client] 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: rgw cache enabled = true 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable ops log = true 2023-01-24T18:57:04.419 INFO:teuthology.orchestra.run.smithi121.stdout: rgw enable usage log = true 2023-01-24T18:57:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-24T18:57:04.420 INFO:teuthology.orchestra.run.smithi121.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-24T18:57:04.420 INFO:teuthology.orchestra.run.smithi121.stdout:[mon.a] 2023-01-24T18:57:04.421 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-24T18:57:04.421 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-24T18:57:04.510 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-24T18:57:04.510 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:04.511 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-24T18:57:04.525 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:04.525 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-24T18:57:04.579 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:57:04.580 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:04.580 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-24T18:57:04.581 DEBUG:teuthology.orchestra.run.smithi121:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-24T18:57:04.652 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:04.652 DEBUG:teuthology.orchestra.run.smithi121:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:04.706 INFO:tasks.ceph:Setting up mon nodes... 2023-01-24T18:57:04.707 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-24T18:57:04.707 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:57:04.792 INFO:teuthology.orchestra.run.smithi121.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-24T18:57:04.794 INFO:tasks.ceph:Setting up mds nodes... 2023-01-24T18:57:04.794 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-24T18:57:04.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-24T18:57:04.831 INFO:teuthology.orchestra.run.smithi121.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-24T18:57:04.843 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-24T18:57:04.843 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi121.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-01-24T18:57:04.844 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-24T18:57:04.907 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:04.907 INFO:tasks.ceph:role: osd.0 2023-01-24T18:57:04.907 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:57:04.908 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-24T18:57:04.978 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:57:04.978 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:57:04.978 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:57:04.978 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-01-24T18:57:04.979 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:57:04.979 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:57:04.979 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:57:04.979 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:57:04.979 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:57:04.980 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:57:05.346 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-24T18:57:05.348 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-24T18:57:05.348 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-24T18:57:05.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-24T18:57:05.512 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-24T18:57:05.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:05.576 INFO:tasks.ceph:role: osd.1 2023-01-24T18:57:05.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:57:05.576 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-24T18:57:05.646 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:57:05.646 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:57:05.647 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:57:05.647 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-01-24T18:57:05.647 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:57:05.647 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:57:05.647 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:57:05.648 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:57:05.648 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:57:05.648 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:57:06.016 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-24T18:57:06.017 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-24T18:57:06.018 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-24T18:57:06.105 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-24T18:57:06.132 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-24T18:57:06.195 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:06.196 INFO:tasks.ceph:role: osd.2 2023-01-24T18:57:06.196 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:57:06.196 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-24T18:57:06.265 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:57:06.266 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:57:06.266 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:57:06.266 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-01-24T18:57:06.266 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:57:06.267 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:57:06.267 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:57:06.267 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:57:06.267 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:57:06.267 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:57:06.637 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-24T18:57:06.638 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-24T18:57:06.638 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-24T18:57:06.741 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-24T18:57:06.768 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-24T18:57:06.832 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:06.832 INFO:tasks.ceph:role: osd.3 2023-01-24T18:57:06.832 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com 2023-01-24T18:57:06.833 DEBUG:teuthology.orchestra.run.smithi121:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-24T18:57:06.904 INFO:teuthology.orchestra.run.smithi121.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-24T18:57:06.904 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-24T18:57:06.904 INFO:teuthology.orchestra.run.smithi121.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-24T18:57:06.905 INFO:teuthology.orchestra.run.smithi121.stdout: = reflink=1 2023-01-24T18:57:06.905 INFO:teuthology.orchestra.run.smithi121.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-24T18:57:06.905 INFO:teuthology.orchestra.run.smithi121.stdout: = sunit=0 swidth=0 blks 2023-01-24T18:57:06.905 INFO:teuthology.orchestra.run.smithi121.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-24T18:57:06.905 INFO:teuthology.orchestra.run.smithi121.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-24T18:57:06.906 INFO:teuthology.orchestra.run.smithi121.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-24T18:57:06.906 INFO:teuthology.orchestra.run.smithi121.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-24T18:57:07.284 INFO:teuthology.orchestra.run.smithi121.stdout:Discarding blocks...Done. 2023-01-24T18:57:07.285 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi121.front.sepia.ceph.com -o noatime 2023-01-24T18:57:07.286 DEBUG:teuthology.orchestra.run.smithi121:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-24T18:57:07.386 DEBUG:teuthology.orchestra.run.smithi121:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-24T18:57:07.413 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-24T18:57:07.477 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.477 INFO:tasks.ceph:role: osd.4 2023-01-24T18:57:07.477 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-24T18:57:07.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.541 INFO:tasks.ceph:role: osd.5 2023-01-24T18:57:07.541 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-24T18:57:07.603 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.604 INFO:tasks.ceph:role: osd.6 2023-01-24T18:57:07.604 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-24T18:57:07.667 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.667 INFO:tasks.ceph:role: osd.7 2023-01-24T18:57:07.667 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2023-01-24T18:57:07.731 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.732 INFO:tasks.ceph:role: osd.8 2023-01-24T18:57:07.732 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2023-01-24T18:57:07.795 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-24T18:57:07.795 INFO:tasks.ceph:role: osd.9 2023-01-24T18:57:07.795 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:07.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.875+0000 7f5de35d6540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-24T18:57:07.877 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.875+0000 7f5de35d6540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-24T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.875+0000 7f5de35d6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.875+0000 7f5de35d6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.875+0000 7f5de35d6540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-24T18:57:07.878 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:07.876+0000 7f5de35d6540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-24T18:57:09.961 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-24T18:57:10.031 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:10.116 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.114+0000 7fd0dfc8b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-24T18:57:10.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.114+0000 7fd0dfc8b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-24T18:57:10.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.114+0000 7fd0dfc8b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:57:10.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.114+0000 7fd0dfc8b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:57:10.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.114+0000 7fd0dfc8b540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-24T18:57:10.117 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:10.115+0000 7fd0dfc8b540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-24T18:57:12.200 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-24T18:57:12.228 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.310+0000 7f699c2ef540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-24T18:57:12.312 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.310+0000 7f699c2ef540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-24T18:57:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.310+0000 7f699c2ef540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:57:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.310+0000 7f699c2ef540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:57:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.310+0000 7f699c2ef540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-24T18:57:12.313 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:12.311+0000 7f699c2ef540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-24T18:57:14.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-24T18:57:14.461 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:14.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.544+0000 7ff381a4f540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-24T18:57:14.546 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.544+0000 7ff381a4f540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-24T18:57:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.544+0000 7ff381a4f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:57:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.544+0000 7ff381a4f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:57:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.544+0000 7ff381a4f540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-24T18:57:14.547 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:14.545+0000 7ff381a4f540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-24T18:57:16.634 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-24T18:57:16.661 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:16.744 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-24T18:57:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-24T18:57:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:57:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:57:16.745 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-24T18:57:16.746 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:16.742+0000 7f5e1e3f2540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-24T18:57:18.822 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-24T18:57:18.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:18.975 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-24T18:57:18.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-24T18:57:18.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:57:18.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:57:18.976 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-24T18:57:18.977 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:18.975+0000 7f771a70c540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-24T18:57:21.055 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-24T18:57:21.083 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:21.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.165+0000 7fbc9147b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-24T18:57:21.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.166+0000 7fbc9147b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-24T18:57:21.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.166+0000 7fbc9147b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:57:21.167 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.166+0000 7fbc9147b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:57:21.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.166+0000 7fbc9147b540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-24T18:57:21.168 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:21.166+0000 7fbc9147b540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-24T18:57:23.244 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-24T18:57:23.313 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:23.397 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.395+0000 7f71852da540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-24T18:57:23.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.395+0000 7f71852da540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-24T18:57:23.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.395+0000 7f71852da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:57:23.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.395+0000 7f71852da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:57:23.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.395+0000 7f71852da540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-24T18:57:23.398 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:23.396+0000 7f71852da540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-24T18:57:25.482 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-24T18:57:25.550 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:25.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.631+0000 7f90e71e4540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2023-01-24T18:57:25.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.631+0000 7f90e71e4540 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2023-01-24T18:57:25.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.631+0000 7f90e71e4540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-24T18:57:25.634 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.631+0000 7f90e71e4540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-24T18:57:25.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.631+0000 7f90e71e4540 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-24T18:57:25.635 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:25.632+0000 7f90e71e4540 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2023-01-24T18:57:27.712 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2023-01-24T18:57:27.779 DEBUG:teuthology.orchestra.run.smithi121:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:27.861 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.859+0000 7f485a58b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2023-01-24T18:57:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.859+0000 7f485a58b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2023-01-24T18:57:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.859+0000 7f485a58b540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-24T18:57:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.859+0000 7f485a58b540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-24T18:57:27.862 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.859+0000 7f485a58b540 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-24T18:57:27.863 INFO:teuthology.orchestra.run.smithi121.stderr:2023-01-24T18:57:27.860+0000 7f485a58b540 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2023-01-24T18:57:29.942 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2023-01-24T18:57:30.009 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-24T18:57:30.009 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.009 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-24T18:57:30.074 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.074 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-24T18:57:30.139 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.139 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-24T18:57:30.203 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.203 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-24T18:57:30.266 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.266 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-24T18:57:30.329 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.329 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-24T18:57:30.392 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.393 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-24T18:57:30.455 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.455 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-24T18:57:30.518 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.519 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-24T18:57:30.581 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.581 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2023-01-24T18:57:30.644 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.645 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2023-01-24T18:57:30.707 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:30.708 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-24T18:57:30.761 INFO:tasks.ceph:Adding keys to all mons... 2023-01-24T18:57:30.762 DEBUG:teuthology.orchestra.run.smithi121:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-24T18:57:30.823 INFO:teuthology.orchestra.run.smithi121.stdout:[mgr.x] 2023-01-24T18:57:30.823 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAAKtBjt8kwLxAAc5LFuuAFifngSt5OQ28DTA== 2023-01-24T18:57:30.824 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.0] 2023-01-24T18:57:30.824 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQADKtBjzBI+NBAAfiwFgXU9ONAH5kwScv2PcA== 2023-01-24T18:57:30.824 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.1] 2023-01-24T18:57:30.825 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAGKtBjjFPjBhAAg8aKsWTBy9bjDq77ZqEQXg== 2023-01-24T18:57:30.825 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.2] 2023-01-24T18:57:30.826 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAIKtBjU8WQEhAAMpe5rnjI0pQHfPvYLRCAzA== 2023-01-24T18:57:30.826 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.3] 2023-01-24T18:57:30.826 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAKKtBjKwiCIBAADBcuEObKlfJvgc0qVPOyRw== 2023-01-24T18:57:30.827 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.4] 2023-01-24T18:57:30.827 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAMKtBjNhZTLBAA7ACxaOonzEsWNAvkdoj6eA== 2023-01-24T18:57:30.827 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.5] 2023-01-24T18:57:30.828 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAOKtBjpi4fOhAAIEU6EvXeULBJbZku33Kpcg== 2023-01-24T18:57:30.828 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.6] 2023-01-24T18:57:30.828 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQARKtBj5lzlCRAA1izzZCmFQ/3rvUgWReBF8g== 2023-01-24T18:57:30.829 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.7] 2023-01-24T18:57:30.829 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQATKtBjGMKmFxAAya1hjQxjo/k9Rpyu2oZT0w== 2023-01-24T18:57:30.830 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.8] 2023-01-24T18:57:30.830 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAVKtBjj6+5JRAAdPk/8Gx7ZyyUmk3BbgAtpg== 2023-01-24T18:57:30.830 INFO:teuthology.orchestra.run.smithi121.stdout:[osd.9] 2023-01-24T18:57:30.831 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAXKtBjkjlPMxAAObsg5XmiWHcJceV4SeILEA== 2023-01-24T18:57:30.831 INFO:teuthology.orchestra.run.smithi121.stdout:[client.0] 2023-01-24T18:57:30.831 INFO:teuthology.orchestra.run.smithi121.stdout: key = AQAAKtBjYSOIMRAA1/U8ic+npuV2dj5F92za6Q== 2023-01-24T18:57:30.833 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-24T18:57:30.918 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.004 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.049 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.095 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.140 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.186 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.233 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.279 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.325 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.371 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-24T18:57:31.417 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-24T18:57:31.465 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-24T18:57:31.466 DEBUG:teuthology.orchestra.run.smithi121:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-24T18:57:31.489 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-24T18:57:31.578 DEBUG:teuthology.orchestra.run.smithi121:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-24T18:57:31.601 DEBUG:teuthology.orchestra.run.smithi121:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-24T18:57:31.655 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-24T18:57:31.656 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-24T18:57:31.656 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-24T18:57:31.698 INFO:tasks.ceph.mon.a:Started 2023-01-24T18:57:31.698 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph config set mgr mgr_pool false --force 2023-01-24T18:57:31.900 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-24T18:57:31.901 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-24T18:57:31.902 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-24T18:57:31.904 INFO:tasks.ceph.mgr.x:Started 2023-01-24T18:57:31.904 INFO:tasks.ceph:Setting crush tunables to default 2023-01-24T18:57:31.905 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd crush tunables default 2023-01-24T18:57:32.103 INFO:teuthology.orchestra.run.smithi121.stderr:adjusted tunables profile to default 2023-01-24T18:57:32.113 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-24T18:57:32.114 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.114 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-24T18:57:32.137 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.138 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-24T18:57:32.201 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.201 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-24T18:57:32.264 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.265 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-24T18:57:32.327 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.328 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-24T18:57:32.390 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.391 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-24T18:57:32.453 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.454 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-24T18:57:32.517 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.518 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-24T18:57:32.581 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.581 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2023-01-24T18:57:32.643 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:57:32.644 DEBUG:teuthology.orchestra.run.smithi121:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2023-01-24T18:57:32.707 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 15954bbe-06b0-4485-9410-769c1c3cb37d 0 2023-01-24T18:57:32.950 INFO:teuthology.orchestra.run.smithi121.stdout:0 2023-01-24T18:57:32.962 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 8327917a-b7e7-4b48-a86b-f964974cb9d8 1 2023-01-24T18:57:33.136 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.134+0000 7efc4e321180 -1 mgr[py] Module not found: 'prometheus' 2023-01-24T18:57:33.137 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.134+0000 7efc4e321180 -1 mgr[py] Traceback (most recent call last): 2023-01-24T18:57:33.137 INFO:tasks.ceph.mgr.x.smithi121.stderr: File "/usr/share/ceph/mgr/prometheus/__init__.py", line 2, in 2023-01-24T18:57:33.137 INFO:tasks.ceph.mgr.x.smithi121.stderr: from .module import Module, StandbyModule 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr: File "/usr/share/ceph/mgr/prometheus/module.py", line 3, in 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr: from packaging.version import Version 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr:ModuleNotFoundError: No module named 'packaging' 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr: 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.135+0000 7efc4e321180 -1 mgr[py] Class not found in module 'prometheus' 2023-01-24T18:57:33.138 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.135+0000 7efc4e321180 -1 mgr[py] Error loading module 'prometheus': (2) No such file or directory 2023-01-24T18:57:33.166 INFO:teuthology.orchestra.run.smithi121.stdout:1 2023-01-24T18:57:33.177 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new 7f00e786-6387-4398-acd5-3b99bb1d11c5 2 2023-01-24T18:57:33.273 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.271+0000 7efc4e321180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-24T18:57:33.351 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.349+0000 7efc4e321180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-24T18:57:33.381 INFO:teuthology.orchestra.run.smithi121.stdout:2 2023-01-24T18:57:33.393 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new ba02d479-684f-4d8e-85c1-bec43226002b 3 2023-01-24T18:57:33.540 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.539+0000 7efc4e321180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:57:33.599 INFO:teuthology.orchestra.run.smithi121.stdout:3 2023-01-24T18:57:33.611 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new dc808a7f-78de-4825-9229-ec4f148b83c0 4 2023-01-24T18:57:33.718 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.716+0000 7efc4e321180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-24T18:57:33.797 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.795+0000 7efc4e321180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-24T18:57:33.815 INFO:teuthology.orchestra.run.smithi121.stdout:4 2023-01-24T18:57:33.828 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new a468112f-5f12-4014-a343-3d202bd83fb9 5 2023-01-24T18:57:33.937 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:33.935+0000 7efc4e321180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-24T18:57:34.032 INFO:teuthology.orchestra.run.smithi121.stdout:5 2023-01-24T18:57:34.034 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:34.032+0000 7efc4e321180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-24T18:57:34.044 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new f38bbe03-38e6-427e-97fe-a73f27fbae78 6 2023-01-24T18:57:34.247 INFO:teuthology.orchestra.run.smithi121.stdout:6 2023-01-24T18:57:34.259 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new ffe4c402-2d51-44f3-b9fd-69191fa98596 7 2023-01-24T18:57:34.475 INFO:teuthology.orchestra.run.smithi121.stdout:7 2023-01-24T18:57:34.487 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new d4940a67-8d4c-4416-bc6a-4cfcc1e7853d 8 2023-01-24T18:57:34.559 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:34.558+0000 7efc4e321180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-24T18:57:34.639 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:34.638+0000 7efc4e321180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-24T18:57:34.690 INFO:teuthology.orchestra.run.smithi121.stdout:8 2023-01-24T18:57:34.703 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd new db696b96-e8bf-4e4f-938a-d6c6d4e2d000 9 2023-01-24T18:57:34.718 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:34.717+0000 7efc4e321180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-24T18:57:34.911 INFO:teuthology.orchestra.run.smithi121.stdout:9 2023-01-24T18:57:34.923 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-24T18:57:34.924 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-24T18:57:34.965 INFO:tasks.ceph.osd.0:Started 2023-01-24T18:57:34.966 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-24T18:57:34.966 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-24T18:57:34.968 INFO:tasks.ceph.osd.1:Started 2023-01-24T18:57:34.969 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-24T18:57:34.969 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-24T18:57:34.971 INFO:tasks.ceph.osd.2:Started 2023-01-24T18:57:34.971 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-24T18:57:34.972 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-24T18:57:34.974 INFO:tasks.ceph.osd.3:Started 2023-01-24T18:57:34.974 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-24T18:57:34.974 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-24T18:57:34.976 INFO:tasks.ceph.osd.4:Started 2023-01-24T18:57:34.977 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-24T18:57:34.977 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-24T18:57:34.979 INFO:tasks.ceph.osd.5:Started 2023-01-24T18:57:34.979 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-24T18:57:34.980 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-24T18:57:34.982 INFO:tasks.ceph.osd.6:Started 2023-01-24T18:57:34.982 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-24T18:57:34.983 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-24T18:57:34.987 INFO:tasks.ceph.osd.7:Started 2023-01-24T18:57:34.987 INFO:tasks.ceph.osd.8:Restarting daemon 2023-01-24T18:57:34.988 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2023-01-24T18:57:34.992 INFO:tasks.ceph.osd.8:Started 2023-01-24T18:57:34.992 INFO:tasks.ceph.osd.9:Restarting daemon 2023-01-24T18:57:34.993 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2023-01-24T18:57:35.001 INFO:tasks.ceph.osd.9:Started 2023-01-24T18:57:35.002 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:57:35.326 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:35.326 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":12,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:57:34.908758+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:57:35.338 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-24T18:57:35.339 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-24T18:57:35.374 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:35.373+0000 7efc4e321180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-24T18:57:35.397 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-01-24T18:57:35.395+0000 7f07d1b12540 -1 Falling back to public interface 2023-01-24T18:57:35.399 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:57:35.397+0000 7fa3277c1540 -1 Falling back to public interface 2023-01-24T18:57:35.402 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:57:35.400+0000 7fabb9725540 -1 Falling back to public interface 2023-01-24T18:57:35.514 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:35.512+0000 7efc4e321180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-24T18:57:35.605 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:35.603+0000 7efc4e321180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-24T18:57:35.646 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:57:35.644+0000 7f4f3ae25540 -1 Falling back to public interface 2023-01-24T18:57:35.646 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:57:35.644+0000 7f86a9fca540 -1 Falling back to public interface 2023-01-24T18:57:35.650 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:57:35.648+0000 7fee5e2c1540 -1 Falling back to public interface 2023-01-24T18:57:35.650 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:57:35.648+0000 7ff37156a540 -1 Falling back to public interface 2023-01-24T18:57:35.654 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:57:35.652+0000 7fa37e459540 -1 Falling back to public interface 2023-01-24T18:57:35.654 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:57:35.652+0000 7fd4e577b540 -1 Falling back to public interface 2023-01-24T18:57:35.708 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:57:35.706+0000 7fdfe07f5540 -1 Falling back to public interface 2023-01-24T18:57:35.853 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:35.851+0000 7efc4e321180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-24T18:57:35.929 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:35.927+0000 7efc4e321180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-24T18:57:36.345 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:57:36.513 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:36.511+0000 7efc4e321180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-24T18:57:36.528 INFO:teuthology.misc.health.smithi121.stdout: 2023-01-24T18:57:36.528 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":12,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:57:34.908758+0000","last_up_change":"0.000000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:57:36.539 DEBUG:teuthology.misc:0 of 10 OSDs are up 2023-01-24T18:57:36.753 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:36.752+0000 7efc4e321180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-24T18:57:36.886 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:36.885+0000 7efc4e321180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-24T18:57:36.970 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:36.968+0000 7efc4e321180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-24T18:57:37.123 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:37.121+0000 7efc4e321180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-24T18:57:37.197 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:37.195+0000 7efc4e321180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-24T18:57:37.272 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:37.270+0000 7efc4e321180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-24T18:57:37.349 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:37.347+0000 7efc4e321180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-24T18:57:37.349 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:37.347+0000 7efc4e321180 -1 log_channel(cluster) log [ERR] : Failed to load ceph-mgr modules: prometheus 2023-01-24T18:57:38.132 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-01-24T18:57:38.130+0000 7f07d1b12540 -1 osd.3 0 log_to_monitors true 2023-01-24T18:57:38.142 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:57:38.141+0000 7fa3277c1540 -1 osd.0 0 log_to_monitors true 2023-01-24T18:57:38.153 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:57:38.152+0000 7fabb9725540 -1 osd.4 0 log_to_monitors true 2023-01-24T18:57:38.346 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:57:38.344+0000 7f4f3ae25540 -1 osd.6 0 log_to_monitors true 2023-01-24T18:57:38.367 INFO:tasks.ceph.mgr.x.smithi121.stderr:2023-01-24T18:57:38.365+0000 7efc077eb700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-01-24T18:57:38.385 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:57:38.383+0000 7fee5e2c1540 -1 osd.1 0 log_to_monitors true 2023-01-24T18:57:38.404 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:57:38.402+0000 7fa37e459540 -1 osd.8 0 log_to_monitors true 2023-01-24T18:57:38.407 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:57:38.405+0000 7f86a9fca540 -1 osd.7 0 log_to_monitors true 2023-01-24T18:57:38.421 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:57:38.419+0000 7ff37156a540 -1 osd.2 0 log_to_monitors true 2023-01-24T18:57:38.423 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:57:38.421+0000 7fd4e577b540 -1 osd.5 0 log_to_monitors true 2023-01-24T18:57:38.446 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:57:38.444+0000 7fdfe07f5540 -1 osd.9 0 log_to_monitors true 2023-01-24T18:57:39.821 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:57:39.819+0000 7f4f31a46700 -1 osd.6 0 waiting for initial osdmap 2023-01-24T18:57:39.822 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-01-24T18:57:39.820+0000 7f07c8733700 -1 osd.3 0 waiting for initial osdmap 2023-01-24T18:57:39.822 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:57:39.821+0000 7fd4dc39c700 -1 osd.5 0 waiting for initial osdmap 2023-01-24T18:57:39.823 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:57:39.819+0000 7fa31e3e2700 -1 osd.0 0 waiting for initial osdmap 2023-01-24T18:57:39.823 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:57:39.821+0000 7f86a0beb700 -1 osd.7 0 waiting for initial osdmap 2023-01-24T18:57:39.824 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:57:39.821+0000 7fa37507a700 -1 osd.8 0 waiting for initial osdmap 2023-01-24T18:57:39.825 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:57:39.823+0000 7fdfd7416700 -1 osd.9 0 waiting for initial osdmap 2023-01-24T18:57:39.826 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:57:39.825+0000 7fee54ee2700 -1 osd.1 0 waiting for initial osdmap 2023-01-24T18:57:39.827 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:57:39.824+0000 7ff36818b700 -1 osd.2 0 waiting for initial osdmap 2023-01-24T18:57:39.827 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:57:39.825+0000 7fabb0346700 -1 osd.4 0 waiting for initial osdmap 2023-01-24T18:57:39.827 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-01-24T18:57:39.826+0000 7f07c1529700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.828 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:57:39.825+0000 7f4f2b03d700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.830 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:57:39.828+0000 7f869a1e2700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.830 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:57:39.829+0000 7fa3171d8700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.831 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:57:39.829+0000 7fdfd0a0d700 -1 osd.9 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.831 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:57:39.830+0000 7fa36e671700 -1 osd.8 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.832 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:57:39.830+0000 7faba913c700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.833 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:57:39.830+0000 7ff361782700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.833 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:57:39.832+0000 7fd4d5993700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:39.834 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:57:39.833+0000 7fee4e4d9700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:57:43.545 DEBUG:teuthology.orchestra.run.smithi121:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-24T18:57:43.847 INFO:teuthology.misc.health.smithi121.stdout: 2023-01-24T18:57:43.848 INFO:teuthology.misc.health.smithi121.stdout:{"epoch":16,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:57:41.845692+0000","last_up_change":"2023-01-24T18:57:40.812714+0000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":86474},{"type":"v1","addr":"172.21.15.121:6809","nonce":86474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":86474},{"type":"v1","addr":"172.21.15.121:6811","nonce":86474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":86474},{"type":"v1","addr":"172.21.15.121:6815","nonce":86474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":86474},{"type":"v1","addr":"172.21.15.121:6813","nonce":86474}]},"public_addr":"172.21.15.121:6809/86474","cluster_addr":"172.21.15.121:6811/86474","heartbeat_back_addr":"172.21.15.121:6815/86474","heartbeat_front_addr":"172.21.15.121:6813/86474","state":["exists","up"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6840","nonce":86476},{"type":"v1","addr":"172.21.15.121:6842","nonce":86476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6844","nonce":86476},{"type":"v1","addr":"172.21.15.121:6846","nonce":86476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6857","nonce":86476},{"type":"v1","addr":"172.21.15.121:6861","nonce":86476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6848","nonce":86476},{"type":"v1","addr":"172.21.15.121:6853","nonce":86476}]},"public_addr":"172.21.15.121:6842/86476","cluster_addr":"172.21.15.121:6846/86476","heartbeat_back_addr":"172.21.15.121:6861/86476","heartbeat_front_addr":"172.21.15.121:6853/86476","state":["exists","up"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6841","nonce":86481},{"type":"v1","addr":"172.21.15.121:6843","nonce":86481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6845","nonce":86481},{"type":"v1","addr":"172.21.15.121:6847","nonce":86481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6855","nonce":86481},{"type":"v1","addr":"172.21.15.121:6859","nonce":86481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6849","nonce":86481},{"type":"v1","addr":"172.21.15.121:6852","nonce":86481}]},"public_addr":"172.21.15.121:6843/86481","cluster_addr":"172.21.15.121:6847/86481","heartbeat_back_addr":"172.21.15.121:6859/86481","heartbeat_front_addr":"172.21.15.121:6852/86481","state":["exists","up"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":86478},{"type":"v1","addr":"172.21.15.121:6801","nonce":86478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":86478},{"type":"v1","addr":"172.21.15.121:6803","nonce":86478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":86478},{"type":"v1","addr":"172.21.15.121:6807","nonce":86478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":86478},{"type":"v1","addr":"172.21.15.121:6805","nonce":86478}]},"public_addr":"172.21.15.121:6801/86478","cluster_addr":"172.21.15.121:6803/86478","heartbeat_back_addr":"172.21.15.121:6807/86478","heartbeat_front_addr":"172.21.15.121:6805/86478","state":["exists","up"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":86515},{"type":"v1","addr":"172.21.15.121:6817","nonce":86515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":86515},{"type":"v1","addr":"172.21.15.121:6819","nonce":86515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":86515},{"type":"v1","addr":"172.21.15.121:6823","nonce":86515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":86515},{"type":"v1","addr":"172.21.15.121:6821","nonce":86515}]},"public_addr":"172.21.15.121:6817/86515","cluster_addr":"172.21.15.121:6819/86515","heartbeat_back_addr":"172.21.15.121:6823/86515","heartbeat_front_addr":"172.21.15.121:6821/86515","state":["exists","up"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6851","nonce":86522},{"type":"v1","addr":"172.21.15.121:6856","nonce":86522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6860","nonce":86522},{"type":"v1","addr":"172.21.15.121:6863","nonce":86522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6869","nonce":86522},{"type":"v1","addr":"172.21.15.121:6871","nonce":86522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6865","nonce":86522},{"type":"v1","addr":"172.21.15.121:6867","nonce":86522}]},"public_addr":"172.21.15.121:6856/86522","cluster_addr":"172.21.15.121:6863/86522","heartbeat_back_addr":"172.21.15.121:6871/86522","heartbeat_front_addr":"172.21.15.121:6867/86522","state":["exists","up"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":86480},{"type":"v1","addr":"172.21.15.121:6826","nonce":86480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":86480},{"type":"v1","addr":"172.21.15.121:6830","nonce":86480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":86480},{"type":"v1","addr":"172.21.15.121:6839","nonce":86480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":86480},{"type":"v1","addr":"172.21.15.121:6834","nonce":86480}]},"public_addr":"172.21.15.121:6826/86480","cluster_addr":"172.21.15.121:6830/86480","heartbeat_back_addr":"172.21.15.121:6839/86480","heartbeat_front_addr":"172.21.15.121:6834/86480","state":["exists","up"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":86494},{"type":"v1","addr":"172.21.15.121:6827","nonce":86494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6829","nonce":86494},{"type":"v1","addr":"172.21.15.121:6831","nonce":86494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6837","nonce":86494},{"type":"v1","addr":"172.21.15.121:6838","nonce":86494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6833","nonce":86494},{"type":"v1","addr":"172.21.15.121:6835","nonce":86494}]},"public_addr":"172.21.15.121:6827/86494","cluster_addr":"172.21.15.121:6831/86494","heartbeat_back_addr":"172.21.15.121:6838/86494","heartbeat_front_addr":"172.21.15.121:6835/86494","state":["exists","up"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6850","nonce":86523},{"type":"v1","addr":"172.21.15.121:6854","nonce":86523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6858","nonce":86523},{"type":"v1","addr":"172.21.15.121:6862","nonce":86523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6868","nonce":86523},{"type":"v1","addr":"172.21.15.121:6870","nonce":86523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6864","nonce":86523},{"type":"v1","addr":"172.21.15.121:6866","nonce":86523}]},"public_addr":"172.21.15.121:6854/86523","cluster_addr":"172.21.15.121:6862/86523","heartbeat_back_addr":"172.21.15.121:6870/86523","heartbeat_front_addr":"172.21.15.121:6866/86523","state":["exists","up"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6872","nonce":86631},{"type":"v1","addr":"172.21.15.121:6873","nonce":86631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6874","nonce":86631},{"type":"v1","addr":"172.21.15.121:6875","nonce":86631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6878","nonce":86631},{"type":"v1","addr":"172.21.15.121:6879","nonce":86631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6876","nonce":86631},{"type":"v1","addr":"172.21.15.121:6877","nonce":86631}]},"public_addr":"172.21.15.121:6873/86631","cluster_addr":"172.21.15.121:6875/86631","heartbeat_back_addr":"172.21.15.121:6879/86631","heartbeat_front_addr":"172.21.15.121:6877/86631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.167472+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.417277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.178997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.379661+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.419300+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.407306+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.459694+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:57:43.860 DEBUG:teuthology.misc:10 of 10 OSDs are up 2023-01-24T18:57:43.860 INFO:tasks.ceph:Creating RBD pool 2023-01-24T18:57:43.861 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-24T18:57:44.869 INFO:teuthology.orchestra.run.smithi121.stderr:pool 'rbd' created 2023-01-24T18:57:44.882 DEBUG:teuthology.orchestra.run.smithi121:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-24T18:57:45.872 INFO:teuthology.orchestra.run.smithi121.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-24T18:57:45.884 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-24T18:57:45.885 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-24T18:57:45.885 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-24T18:57:45.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-24T18:57:45.886 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-24T18:57:46.238 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:46.250 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6880","nonce":85649},{"type":"v1","addr":"172.21.15.121:6881","nonce":85649}]},"active_addr":"172.21.15.121:6881/85649","active_change":"2023-01-24T18:57:37.360881+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":false,"error_string":"No module named 'packaging'","module_options":{}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3946055780}]},{"addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3550483564}]},{"addrvec":[{"type":"v2","addr":"172.21.15.121:0","nonce":3504334306}]}]}} 2023-01-24T18:57:46.252 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-24T18:57:46.253 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-24T18:57:46.253 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:57:46.554 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:46.554 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":18,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:57:45.863496+0000","last_up_change":"2023-01-24T18:57:40.812714+0000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T18:57:44.174050+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":86474},{"type":"v1","addr":"172.21.15.121:6809","nonce":86474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":86474},{"type":"v1","addr":"172.21.15.121:6811","nonce":86474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":86474},{"type":"v1","addr":"172.21.15.121:6815","nonce":86474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":86474},{"type":"v1","addr":"172.21.15.121:6813","nonce":86474}]},"public_addr":"172.21.15.121:6809/86474","cluster_addr":"172.21.15.121:6811/86474","heartbeat_back_addr":"172.21.15.121:6815/86474","heartbeat_front_addr":"172.21.15.121:6813/86474","state":["exists","up"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6840","nonce":86476},{"type":"v1","addr":"172.21.15.121:6842","nonce":86476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6844","nonce":86476},{"type":"v1","addr":"172.21.15.121:6846","nonce":86476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6857","nonce":86476},{"type":"v1","addr":"172.21.15.121:6861","nonce":86476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6848","nonce":86476},{"type":"v1","addr":"172.21.15.121:6853","nonce":86476}]},"public_addr":"172.21.15.121:6842/86476","cluster_addr":"172.21.15.121:6846/86476","heartbeat_back_addr":"172.21.15.121:6861/86476","heartbeat_front_addr":"172.21.15.121:6853/86476","state":["exists","up"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6841","nonce":86481},{"type":"v1","addr":"172.21.15.121:6843","nonce":86481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6845","nonce":86481},{"type":"v1","addr":"172.21.15.121:6847","nonce":86481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6855","nonce":86481},{"type":"v1","addr":"172.21.15.121:6859","nonce":86481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6849","nonce":86481},{"type":"v1","addr":"172.21.15.121:6852","nonce":86481}]},"public_addr":"172.21.15.121:6843/86481","cluster_addr":"172.21.15.121:6847/86481","heartbeat_back_addr":"172.21.15.121:6859/86481","heartbeat_front_addr":"172.21.15.121:6852/86481","state":["exists","up"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":86478},{"type":"v1","addr":"172.21.15.121:6801","nonce":86478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":86478},{"type":"v1","addr":"172.21.15.121:6803","nonce":86478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":86478},{"type":"v1","addr":"172.21.15.121:6807","nonce":86478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":86478},{"type":"v1","addr":"172.21.15.121:6805","nonce":86478}]},"public_addr":"172.21.15.121:6801/86478","cluster_addr":"172.21.15.121:6803/86478","heartbeat_back_addr":"172.21.15.121:6807/86478","heartbeat_front_addr":"172.21.15.121:6805/86478","state":["exists","up"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":86515},{"type":"v1","addr":"172.21.15.121:6817","nonce":86515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":86515},{"type":"v1","addr":"172.21.15.121:6819","nonce":86515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":86515},{"type":"v1","addr":"172.21.15.121:6823","nonce":86515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":86515},{"type":"v1","addr":"172.21.15.121:6821","nonce":86515}]},"public_addr":"172.21.15.121:6817/86515","cluster_addr":"172.21.15.121:6819/86515","heartbeat_back_addr":"172.21.15.121:6823/86515","heartbeat_front_addr":"172.21.15.121:6821/86515","state":["exists","up"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6851","nonce":86522},{"type":"v1","addr":"172.21.15.121:6856","nonce":86522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6860","nonce":86522},{"type":"v1","addr":"172.21.15.121:6863","nonce":86522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6869","nonce":86522},{"type":"v1","addr":"172.21.15.121:6871","nonce":86522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6865","nonce":86522},{"type":"v1","addr":"172.21.15.121:6867","nonce":86522}]},"public_addr":"172.21.15.121:6856/86522","cluster_addr":"172.21.15.121:6863/86522","heartbeat_back_addr":"172.21.15.121:6871/86522","heartbeat_front_addr":"172.21.15.121:6867/86522","state":["exists","up"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":86480},{"type":"v1","addr":"172.21.15.121:6826","nonce":86480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":86480},{"type":"v1","addr":"172.21.15.121:6830","nonce":86480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":86480},{"type":"v1","addr":"172.21.15.121:6839","nonce":86480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":86480},{"type":"v1","addr":"172.21.15.121:6834","nonce":86480}]},"public_addr":"172.21.15.121:6826/86480","cluster_addr":"172.21.15.121:6830/86480","heartbeat_back_addr":"172.21.15.121:6839/86480","heartbeat_front_addr":"172.21.15.121:6834/86480","state":["exists","up"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":86494},{"type":"v1","addr":"172.21.15.121:6827","nonce":86494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6829","nonce":86494},{"type":"v1","addr":"172.21.15.121:6831","nonce":86494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6837","nonce":86494},{"type":"v1","addr":"172.21.15.121:6838","nonce":86494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6833","nonce":86494},{"type":"v1","addr":"172.21.15.121:6835","nonce":86494}]},"public_addr":"172.21.15.121:6827/86494","cluster_addr":"172.21.15.121:6831/86494","heartbeat_back_addr":"172.21.15.121:6838/86494","heartbeat_front_addr":"172.21.15.121:6835/86494","state":["exists","up"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6850","nonce":86523},{"type":"v1","addr":"172.21.15.121:6854","nonce":86523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6858","nonce":86523},{"type":"v1","addr":"172.21.15.121:6862","nonce":86523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6868","nonce":86523},{"type":"v1","addr":"172.21.15.121:6870","nonce":86523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6864","nonce":86523},{"type":"v1","addr":"172.21.15.121:6866","nonce":86523}]},"public_addr":"172.21.15.121:6854/86523","cluster_addr":"172.21.15.121:6862/86523","heartbeat_back_addr":"172.21.15.121:6870/86523","heartbeat_front_addr":"172.21.15.121:6866/86523","state":["exists","up"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6872","nonce":86631},{"type":"v1","addr":"172.21.15.121:6873","nonce":86631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6874","nonce":86631},{"type":"v1","addr":"172.21.15.121:6875","nonce":86631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6878","nonce":86631},{"type":"v1","addr":"172.21.15.121:6879","nonce":86631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6876","nonce":86631},{"type":"v1","addr":"172.21.15.121:6877","nonce":86631}]},"public_addr":"172.21.15.121:6873/86631","cluster_addr":"172.21.15.121:6875/86631","heartbeat_back_addr":"172.21.15.121:6879/86631","heartbeat_front_addr":"172.21.15.121:6877/86631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.167472+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.417277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.178997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.379661+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.419300+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.407306+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.459694+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:57:46.567 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-24T18:57:46.568 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:57:46.871 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:46.871 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":19,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:57:46.865481+0000","last_up_change":"2023-01-24T18:57:40.812714+0000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T18:57:44.174050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":86474},{"type":"v1","addr":"172.21.15.121:6809","nonce":86474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6810","nonce":86474},{"type":"v1","addr":"172.21.15.121:6811","nonce":86474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":86474},{"type":"v1","addr":"172.21.15.121:6815","nonce":86474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":86474},{"type":"v1","addr":"172.21.15.121:6813","nonce":86474}]},"public_addr":"172.21.15.121:6809/86474","cluster_addr":"172.21.15.121:6811/86474","heartbeat_back_addr":"172.21.15.121:6815/86474","heartbeat_front_addr":"172.21.15.121:6813/86474","state":["exists","up"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6840","nonce":86476},{"type":"v1","addr":"172.21.15.121:6842","nonce":86476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6844","nonce":86476},{"type":"v1","addr":"172.21.15.121:6846","nonce":86476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6857","nonce":86476},{"type":"v1","addr":"172.21.15.121:6861","nonce":86476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6848","nonce":86476},{"type":"v1","addr":"172.21.15.121:6853","nonce":86476}]},"public_addr":"172.21.15.121:6842/86476","cluster_addr":"172.21.15.121:6846/86476","heartbeat_back_addr":"172.21.15.121:6861/86476","heartbeat_front_addr":"172.21.15.121:6853/86476","state":["exists","up"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6841","nonce":86481},{"type":"v1","addr":"172.21.15.121:6843","nonce":86481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6845","nonce":86481},{"type":"v1","addr":"172.21.15.121:6847","nonce":86481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6855","nonce":86481},{"type":"v1","addr":"172.21.15.121:6859","nonce":86481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6849","nonce":86481},{"type":"v1","addr":"172.21.15.121:6852","nonce":86481}]},"public_addr":"172.21.15.121:6843/86481","cluster_addr":"172.21.15.121:6847/86481","heartbeat_back_addr":"172.21.15.121:6859/86481","heartbeat_front_addr":"172.21.15.121:6852/86481","state":["exists","up"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":86478},{"type":"v1","addr":"172.21.15.121:6801","nonce":86478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":86478},{"type":"v1","addr":"172.21.15.121:6803","nonce":86478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":86478},{"type":"v1","addr":"172.21.15.121:6807","nonce":86478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":86478},{"type":"v1","addr":"172.21.15.121:6805","nonce":86478}]},"public_addr":"172.21.15.121:6801/86478","cluster_addr":"172.21.15.121:6803/86478","heartbeat_back_addr":"172.21.15.121:6807/86478","heartbeat_front_addr":"172.21.15.121:6805/86478","state":["exists","up"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":86515},{"type":"v1","addr":"172.21.15.121:6817","nonce":86515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":86515},{"type":"v1","addr":"172.21.15.121:6819","nonce":86515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":86515},{"type":"v1","addr":"172.21.15.121:6823","nonce":86515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":86515},{"type":"v1","addr":"172.21.15.121:6821","nonce":86515}]},"public_addr":"172.21.15.121:6817/86515","cluster_addr":"172.21.15.121:6819/86515","heartbeat_back_addr":"172.21.15.121:6823/86515","heartbeat_front_addr":"172.21.15.121:6821/86515","state":["exists","up"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6851","nonce":86522},{"type":"v1","addr":"172.21.15.121:6856","nonce":86522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6860","nonce":86522},{"type":"v1","addr":"172.21.15.121:6863","nonce":86522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6869","nonce":86522},{"type":"v1","addr":"172.21.15.121:6871","nonce":86522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6865","nonce":86522},{"type":"v1","addr":"172.21.15.121:6867","nonce":86522}]},"public_addr":"172.21.15.121:6856/86522","cluster_addr":"172.21.15.121:6863/86522","heartbeat_back_addr":"172.21.15.121:6871/86522","heartbeat_front_addr":"172.21.15.121:6867/86522","state":["exists","up"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":86480},{"type":"v1","addr":"172.21.15.121:6826","nonce":86480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":86480},{"type":"v1","addr":"172.21.15.121:6830","nonce":86480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":86480},{"type":"v1","addr":"172.21.15.121:6839","nonce":86480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":86480},{"type":"v1","addr":"172.21.15.121:6834","nonce":86480}]},"public_addr":"172.21.15.121:6826/86480","cluster_addr":"172.21.15.121:6830/86480","heartbeat_back_addr":"172.21.15.121:6839/86480","heartbeat_front_addr":"172.21.15.121:6834/86480","state":["exists","up"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":86494},{"type":"v1","addr":"172.21.15.121:6827","nonce":86494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6829","nonce":86494},{"type":"v1","addr":"172.21.15.121:6831","nonce":86494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6837","nonce":86494},{"type":"v1","addr":"172.21.15.121:6838","nonce":86494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6833","nonce":86494},{"type":"v1","addr":"172.21.15.121:6835","nonce":86494}]},"public_addr":"172.21.15.121:6827/86494","cluster_addr":"172.21.15.121:6831/86494","heartbeat_back_addr":"172.21.15.121:6838/86494","heartbeat_front_addr":"172.21.15.121:6835/86494","state":["exists","up"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6850","nonce":86523},{"type":"v1","addr":"172.21.15.121:6854","nonce":86523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6858","nonce":86523},{"type":"v1","addr":"172.21.15.121:6862","nonce":86523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6868","nonce":86523},{"type":"v1","addr":"172.21.15.121:6870","nonce":86523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6864","nonce":86523},{"type":"v1","addr":"172.21.15.121:6866","nonce":86523}]},"public_addr":"172.21.15.121:6854/86523","cluster_addr":"172.21.15.121:6862/86523","heartbeat_back_addr":"172.21.15.121:6870/86523","heartbeat_front_addr":"172.21.15.121:6866/86523","state":["exists","up"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6872","nonce":86631},{"type":"v1","addr":"172.21.15.121:6873","nonce":86631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6874","nonce":86631},{"type":"v1","addr":"172.21.15.121:6875","nonce":86631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6878","nonce":86631},{"type":"v1","addr":"172.21.15.121:6879","nonce":86631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6876","nonce":86631},{"type":"v1","addr":"172.21.15.121:6877","nonce":86631}]},"public_addr":"172.21.15.121:6873/86631","cluster_addr":"172.21.15.121:6875/86631","heartbeat_back_addr":"172.21.15.121:6879/86631","heartbeat_front_addr":"172.21.15.121:6877/86631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.167472+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388192+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.417277+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.172991+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.178997+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.379661+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388286+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.419300+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.407306+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.459694+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:57:46.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-24T18:57:46.885 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-24T18:57:46.886 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-24T18:57:46.886 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-24T18:57:46.887 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-24T18:57:46.888 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-24T18:57:46.888 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-24T18:57:46.889 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-24T18:57:46.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2023-01-24T18:57:46.890 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2023-01-24T18:57:47.204 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.204 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-24T18:57:47.205 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.206 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2023-01-24T18:57:47.209 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.210 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:57:47.211 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.212 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-24T18:57:47.214 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.214 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-24T18:57:47.226 INFO:teuthology.orchestra.run.smithi121.stdout:64424509444 2023-01-24T18:57:47.227 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2023-01-24T18:57:47.232 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.232 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-24T18:57:47.237 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.237 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-24T18:57:47.248 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.248 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-24T18:57:47.286 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.286 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-24T18:57:47.885 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.888 INFO:teuthology.orchestra.run.smithi121.stdout:64424509441 2023-01-24T18:57:47.889 INFO:teuthology.orchestra.run.smithi121.stdout:64424509444 2023-01-24T18:57:47.889 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.2 2023-01-24T18:57:47.911 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.4 2023-01-24T18:57:47.917 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.932 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.937 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.9 2023-01-24T18:57:47.938 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509441 for osd.6 2023-01-24T18:57:47.956 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:47.964 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.1 2023-01-24T18:57:47.964 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.965 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.5 2023-01-24T18:57:47.966 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.977 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.8 2023-01-24T18:57:47.977 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:47.994 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:48.008 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.7 2023-01-24T18:57:48.009 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:48.035 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:48.049 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.0 2023-01-24T18:57:48.049 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:48.076 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:48.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.3 2023-01-24T18:57:48.090 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:48.947 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-24T18:57:49.263 INFO:teuthology.orchestra.run.smithi121.stdout:64424509443 2023-01-24T18:57:49.276 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.6 2023-01-24T18:57:49.276 DEBUG:teuthology.parallel:result is None 2023-01-24T18:57:49.277 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:57:49.277 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:57:49.575 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:49.577 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:57:49.589 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-01-24T18:57:49.368321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":10,"num_per_pool_osds":9,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":50416,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":48704,"kb_avail":1048525584,"statfs":{"total":1073741824000,"available":1073690198016,"internally_reserved":0,"allocated":1671168,"data_stored":506409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":49872896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.484591+0000","last_change":"2023-01-24T18:57:46.484591+0000","last_active":"2023-01-24T18:57:46.484591+0000","last_peered":"2023-01-24T18:57:46.484591+0000","last_clean":"2023-01-24T18:57:46.484591+0000","last_became_active":"2023-01-24T18:57:46.484371+0000","last_became_peered":"2023-01-24T18:57:46.484371+0000","last_unstale":"2023-01-24T18:57:46.484591+0000","last_undegraded":"2023-01-24T18:57:46.484591+0000","last_fullsized":"2023-01-24T18:57:46.484591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:08:32.880685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.337816+0000","last_change":"2023-01-24T18:57:46.337816+0000","last_active":"2023-01-24T18:57:46.337816+0000","last_peered":"2023-01-24T18:57:46.337816+0000","last_clean":"2023-01-24T18:57:46.337816+0000","last_became_active":"2023-01-24T18:57:46.337566+0000","last_became_peered":"2023-01-24T18:57:46.337566+0000","last_unstale":"2023-01-24T18:57:46.337816+0000","last_undegraded":"2023-01-24T18:57:46.337816+0000","last_fullsized":"2023-01-24T18:57:46.337816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:45:08.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.871811+0000","last_change":"2023-01-24T18:57:45.871811+0000","last_active":"2023-01-24T18:57:45.871811+0000","last_peered":"2023-01-24T18:57:45.871811+0000","last_clean":"2023-01-24T18:57:45.871811+0000","last_became_active":"2023-01-24T18:57:45.871567+0000","last_became_peered":"2023-01-24T18:57:45.871567+0000","last_unstale":"2023-01-24T18:57:45.871811+0000","last_undegraded":"2023-01-24T18:57:45.871811+0000","last_fullsized":"2023-01-24T18:57:45.871811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:57:33.561016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.877032+0000","last_change":"2023-01-24T18:57:45.877032+0000","last_active":"2023-01-24T18:57:45.877032+0000","last_peered":"2023-01-24T18:57:45.877032+0000","last_clean":"2023-01-24T18:57:45.877032+0000","last_became_active":"2023-01-24T18:57:45.876782+0000","last_became_peered":"2023-01-24T18:57:45.876782+0000","last_unstale":"2023-01-24T18:57:45.877032+0000","last_undegraded":"2023-01-24T18:57:45.877032+0000","last_fullsized":"2023-01-24T18:57:45.877032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:33:11.137094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.484730+0000","last_change":"2023-01-24T18:57:46.484730+0000","last_active":"2023-01-24T18:57:46.484730+0000","last_peered":"2023-01-24T18:57:46.484730+0000","last_clean":"2023-01-24T18:57:46.484730+0000","last_became_active":"2023-01-24T18:57:46.484468+0000","last_became_peered":"2023-01-24T18:57:46.484468+0000","last_unstale":"2023-01-24T18:57:46.484730+0000","last_undegraded":"2023-01-24T18:57:46.484730+0000","last_fullsized":"2023-01-24T18:57:46.484730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:32:34.333150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.496126+0000","last_change":"2023-01-24T18:57:46.496126+0000","last_active":"2023-01-24T18:57:46.496126+0000","last_peered":"2023-01-24T18:57:46.496126+0000","last_clean":"2023-01-24T18:57:46.496126+0000","last_became_active":"2023-01-24T18:57:46.495857+0000","last_became_peered":"2023-01-24T18:57:46.495857+0000","last_unstale":"2023-01-24T18:57:46.496126+0000","last_undegraded":"2023-01-24T18:57:46.496126+0000","last_fullsized":"2023-01-24T18:57:46.496126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:16:13.508480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.877488+0000","last_change":"2023-01-24T18:57:45.877227+0000","last_active":"2023-01-24T18:57:45.877488+0000","last_peered":"2023-01-24T18:57:45.877488+0000","last_clean":"2023-01-24T18:57:45.877488+0000","last_became_active":"2023-01-24T18:57:45.876781+0000","last_became_peered":"2023-01-24T18:57:45.876781+0000","last_unstale":"2023-01-24T18:57:45.877488+0000","last_undegraded":"2023-01-24T18:57:45.877488+0000","last_fullsized":"2023-01-24T18:57:45.877488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:32:34.333150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.874509+0000","last_change":"2023-01-24T18:57:45.874509+0000","last_active":"2023-01-24T18:57:45.874509+0000","last_peered":"2023-01-24T18:57:45.874509+0000","last_clean":"2023-01-24T18:57:45.874509+0000","last_became_active":"2023-01-24T18:57:45.874242+0000","last_became_peered":"2023-01-24T18:57:45.874242+0000","last_unstale":"2023-01-24T18:57:45.874509+0000","last_undegraded":"2023-01-24T18:57:45.874509+0000","last_fullsized":"2023-01-24T18:57:45.874509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:51:22.822314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5104,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":104852496,"statfs":{"total":107374182400,"available":107368955904,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852584,"statfs":{"total":107374182400,"available":107369046016,"internally_reserved":0,"allocated":147456,"data_stored":40834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5052,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852548,"statfs":{"total":107374182400,"available":107369009152,"internally_reserved":0,"allocated":184320,"data_stored":59207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:57:49.592 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:57:49.889 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:49.891 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:57:49.901 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2023-01-24T18:57:49.368321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":10,"num_per_pool_osds":9,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":50416,"kb_used_data":1632,"kb_used_omap":0,"kb_used_meta":48704,"kb_avail":1048525584,"statfs":{"total":1073741824000,"available":1073690198016,"internally_reserved":0,"allocated":1671168,"data_stored":506409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":49872896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.484591+0000","last_change":"2023-01-24T18:57:46.484591+0000","last_active":"2023-01-24T18:57:46.484591+0000","last_peered":"2023-01-24T18:57:46.484591+0000","last_clean":"2023-01-24T18:57:46.484591+0000","last_became_active":"2023-01-24T18:57:46.484371+0000","last_became_peered":"2023-01-24T18:57:46.484371+0000","last_unstale":"2023-01-24T18:57:46.484591+0000","last_undegraded":"2023-01-24T18:57:46.484591+0000","last_fullsized":"2023-01-24T18:57:46.484591+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:08:32.880685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.337816+0000","last_change":"2023-01-24T18:57:46.337816+0000","last_active":"2023-01-24T18:57:46.337816+0000","last_peered":"2023-01-24T18:57:46.337816+0000","last_clean":"2023-01-24T18:57:46.337816+0000","last_became_active":"2023-01-24T18:57:46.337566+0000","last_became_peered":"2023-01-24T18:57:46.337566+0000","last_unstale":"2023-01-24T18:57:46.337816+0000","last_undegraded":"2023-01-24T18:57:46.337816+0000","last_fullsized":"2023-01-24T18:57:46.337816+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:45:08.827026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.871811+0000","last_change":"2023-01-24T18:57:45.871811+0000","last_active":"2023-01-24T18:57:45.871811+0000","last_peered":"2023-01-24T18:57:45.871811+0000","last_clean":"2023-01-24T18:57:45.871811+0000","last_became_active":"2023-01-24T18:57:45.871567+0000","last_became_peered":"2023-01-24T18:57:45.871567+0000","last_unstale":"2023-01-24T18:57:45.871811+0000","last_undegraded":"2023-01-24T18:57:45.871811+0000","last_fullsized":"2023-01-24T18:57:45.871811+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T22:57:33.561016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.877032+0000","last_change":"2023-01-24T18:57:45.877032+0000","last_active":"2023-01-24T18:57:45.877032+0000","last_peered":"2023-01-24T18:57:45.877032+0000","last_clean":"2023-01-24T18:57:45.877032+0000","last_became_active":"2023-01-24T18:57:45.876782+0000","last_became_peered":"2023-01-24T18:57:45.876782+0000","last_unstale":"2023-01-24T18:57:45.877032+0000","last_undegraded":"2023-01-24T18:57:45.877032+0000","last_fullsized":"2023-01-24T18:57:45.877032+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:33:11.137094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.484730+0000","last_change":"2023-01-24T18:57:46.484730+0000","last_active":"2023-01-24T18:57:46.484730+0000","last_peered":"2023-01-24T18:57:46.484730+0000","last_clean":"2023-01-24T18:57:46.484730+0000","last_became_active":"2023-01-24T18:57:46.484468+0000","last_became_peered":"2023-01-24T18:57:46.484468+0000","last_unstale":"2023-01-24T18:57:46.484730+0000","last_undegraded":"2023-01-24T18:57:46.484730+0000","last_fullsized":"2023-01-24T18:57:46.484730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:32:34.333150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:46.496126+0000","last_change":"2023-01-24T18:57:46.496126+0000","last_active":"2023-01-24T18:57:46.496126+0000","last_peered":"2023-01-24T18:57:46.496126+0000","last_clean":"2023-01-24T18:57:46.496126+0000","last_became_active":"2023-01-24T18:57:46.495857+0000","last_became_peered":"2023-01-24T18:57:46.495857+0000","last_unstale":"2023-01-24T18:57:46.496126+0000","last_undegraded":"2023-01-24T18:57:46.496126+0000","last_fullsized":"2023-01-24T18:57:46.496126+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:16:13.508480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.877488+0000","last_change":"2023-01-24T18:57:45.877227+0000","last_active":"2023-01-24T18:57:45.877488+0000","last_peered":"2023-01-24T18:57:45.877488+0000","last_clean":"2023-01-24T18:57:45.877488+0000","last_became_active":"2023-01-24T18:57:45.876781+0000","last_became_peered":"2023-01-24T18:57:45.876781+0000","last_unstale":"2023-01-24T18:57:45.877488+0000","last_undegraded":"2023-01-24T18:57:45.877488+0000","last_fullsized":"2023-01-24T18:57:45.877488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:32:34.333150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":8,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-24T18:57:45.874509+0000","last_change":"2023-01-24T18:57:45.874509+0000","last_active":"2023-01-24T18:57:45.874509+0000","last_peered":"2023-01-24T18:57:45.874509+0000","last_clean":"2023-01-24T18:57:45.874509+0000","last_became_active":"2023-01-24T18:57:45.874242+0000","last_became_peered":"2023-01-24T18:57:45.874242+0000","last_unstale":"2023-01-24T18:57:45.874509+0000","last_undegraded":"2023-01-24T18:57:45.874509+0000","last_fullsized":"2023-01-24T18:57:45.874509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T20:51:22.822314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.155}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67200000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72699999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.079}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94699999999999995}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]}]},{"osd":8,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0820000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.26}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76500000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1990000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.641}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63400000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5129999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2509999999999999}]}]},{"osd":6,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69599999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.091}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.121}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.274}]}]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5028,"kb_used_data":156,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852572,"statfs":{"total":107374182400,"available":107369033728,"internally_reserved":0,"allocated":159744,"data_stored":46951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41299999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.4710000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5800000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95399999999999996}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5104,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":104852496,"statfs":{"total":107374182400,"available":107368955904,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]},{"osd":2,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5016,"kb_used_data":144,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852584,"statfs":{"total":107374182400,"available":107369046016,"internally_reserved":0,"allocated":147456,"data_stored":40834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95999999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66700000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1419999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0609999999999999}]}]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5052,"kb_used_data":180,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852548,"statfs":{"total":107374182400,"available":107369009152,"internally_reserved":0,"allocated":184320,"data_stored":59207,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84299999999999997}]}]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.177}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85099999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.21}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:57:49.903 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-24T18:57:49.903 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-24T18:57:49.904 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-24T18:57:49.904 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-24T18:57:50.273 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:57:50.273 INFO:teuthology.orchestra.run.smithi121.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-24T18:57:50.287 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-24T18:57:50.288 INFO:teuthology.run_tasks:Running task workunit... 2023-01-24T18:57:50.301 INFO:tasks.workunit:Pulling workunits from ref 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:57:50.301 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-01-24T18:57:50.301 INFO:tasks.workunit:timeout=3h 2023-01-24T18:57:50.302 INFO:tasks.workunit:cleanup=True 2023-01-24T18:57:50.302 DEBUG:teuthology.orchestra.run.smithi121:> stat -- /home/ubuntu/cephtest/mnt.0 2023-01-24T18:57:50.316 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-24T18:57:50.316 INFO:teuthology.orchestra.run.smithi121.stderr:stat: cannot stat '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-01-24T18:57:50.317 DEBUG:teuthology.orchestra.run.smithi121:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-01-24T18:57:50.371 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-01-24T18:57:50.371 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-01-24T18:57:50.425 DEBUG:teuthology.orchestra.run.smithi121:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 49f8fb05584886826e8eade75f7105fba754560c 2023-01-24T18:57:50.479 INFO:tasks.workunit.client.0.smithi121.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-01-24T18:59:00.875 INFO:tasks.workunit.client.0.smithi121.stderr:Note: switching to '49f8fb05584886826e8eade75f7105fba754560c'. 2023-01-24T18:59:00.875 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.876 INFO:tasks.workunit.client.0.smithi121.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-01-24T18:59:00.876 INFO:tasks.workunit.client.0.smithi121.stderr:changes and commit them, and you can discard any commits you make in this 2023-01-24T18:59:00.876 INFO:tasks.workunit.client.0.smithi121.stderr:state without impacting any branches by switching back to a branch. 2023-01-24T18:59:00.876 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.877 INFO:tasks.workunit.client.0.smithi121.stderr:If you want to create a new branch to retain commits you create, you may 2023-01-24T18:59:00.877 INFO:tasks.workunit.client.0.smithi121.stderr:do so (now or later) by using -c with the switch command. Example: 2023-01-24T18:59:00.877 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.877 INFO:tasks.workunit.client.0.smithi121.stderr: git switch -c 2023-01-24T18:59:00.878 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.878 INFO:tasks.workunit.client.0.smithi121.stderr:Or undo this operation with: 2023-01-24T18:59:00.878 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.878 INFO:tasks.workunit.client.0.smithi121.stderr: git switch - 2023-01-24T18:59:00.878 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.879 INFO:tasks.workunit.client.0.smithi121.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-01-24T18:59:00.879 INFO:tasks.workunit.client.0.smithi121.stderr: 2023-01-24T18:59:00.879 INFO:tasks.workunit.client.0.smithi121.stderr:HEAD is now at 49f8fb05584 Merge branch 'wip-58543-quincy' of https://github.com/ivancich/ceph-fork into wip-yuri7-testing-2023-01-23-1532-quincy 2023-01-24T18:59:00.880 DEBUG:teuthology.orchestra.run.smithi121:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-01-24T18:59:00.955 INFO:tasks.workunit.client.0.smithi121.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-01-24T18:59:00.957 INFO:tasks.workunit.client.0.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T18:59:00.957 INFO:tasks.workunit.client.0.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-01-24T18:59:01.573 INFO:tasks.workunit.client.0.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-01-24T18:59:01.636 INFO:tasks.workunit.client.0.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-01-24T18:59:01.673 INFO:tasks.workunit.client.0.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-24T18:59:01.674 INFO:tasks.workunit.client.0.smithi121.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T18:59:01.674 INFO:tasks.workunit.client.0.smithi121.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-01-24T18:59:01.710 INFO:tasks.workunit.client.0.smithi121.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-24T18:59:01.712 DEBUG:teuthology.orchestra.run.smithi121:> set -ex 2023-01-24T18:59:01.713 DEBUG:teuthology.orchestra.run.smithi121:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-01-24T18:59:01.767 INFO:tasks.workunit:Running workunits matching rados/test_health_warnings.sh on client.0... 2023-01-24T18:59:01.769 INFO:tasks.workunit:Running workunit rados/test_health_warnings.sh... 2023-01-24T18:59:01.769 DEBUG:teuthology.orchestra.run.smithi121:workunit test rados/test_health_warnings.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=49f8fb05584886826e8eade75f7105fba754560c TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_health_warnings.sh 2023-01-24T18:59:01.825 INFO:tasks.workunit.client.0.smithi121.stderr:+ crushtool -o crushmap --build --num_osds 10 host straw 2 rack straw 2 row straw 2 root straw 0 2023-01-24T18:59:01.839 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd setcrushmap -i crushmap 2023-01-24T18:59:03.947 INFO:tasks.workunit.client.0.smithi121.stderr:6 2023-01-24T18:59:03.961 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd tree 2023-01-24T18:59:04.362 INFO:tasks.workunit.client.0.smithi121.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2023-01-24T18:59:04.363 INFO:tasks.workunit.client.0.smithi121.stdout:-11 10.00000 root root 2023-01-24T18:59:04.363 INFO:tasks.workunit.client.0.smithi121.stdout: -9 8.00000 row row0 2023-01-24T18:59:04.363 INFO:tasks.workunit.client.0.smithi121.stdout: -6 4.00000 rack rack0 2023-01-24T18:59:04.363 INFO:tasks.workunit.client.0.smithi121.stdout: -1 2.00000 host host0 2023-01-24T18:59:04.363 INFO:tasks.workunit.client.0.smithi121.stdout: 0 1.00000 osd.0 up 1.00000 1.00000 2023-01-24T18:59:04.364 INFO:tasks.workunit.client.0.smithi121.stdout: 1 1.00000 osd.1 up 1.00000 1.00000 2023-01-24T18:59:04.364 INFO:tasks.workunit.client.0.smithi121.stdout: -2 2.00000 host host1 2023-01-24T18:59:04.364 INFO:tasks.workunit.client.0.smithi121.stdout: 2 1.00000 osd.2 up 1.00000 1.00000 2023-01-24T18:59:04.364 INFO:tasks.workunit.client.0.smithi121.stdout: 3 1.00000 osd.3 up 1.00000 1.00000 2023-01-24T18:59:04.365 INFO:tasks.workunit.client.0.smithi121.stdout: -7 4.00000 rack rack1 2023-01-24T18:59:04.365 INFO:tasks.workunit.client.0.smithi121.stdout: -3 2.00000 host host2 2023-01-24T18:59:04.365 INFO:tasks.workunit.client.0.smithi121.stdout: 4 1.00000 osd.4 up 1.00000 1.00000 2023-01-24T18:59:04.365 INFO:tasks.workunit.client.0.smithi121.stdout: 5 1.00000 osd.5 up 1.00000 1.00000 2023-01-24T18:59:04.365 INFO:tasks.workunit.client.0.smithi121.stdout: -4 2.00000 host host3 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout: 6 1.00000 osd.6 up 1.00000 1.00000 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout: 7 1.00000 osd.7 up 1.00000 1.00000 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout:-10 2.00000 row row1 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout: -8 2.00000 rack rack2 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout: -5 2.00000 host host4 2023-01-24T18:59:04.366 INFO:tasks.workunit.client.0.smithi121.stdout: 8 1.00000 osd.8 up 1.00000 1.00000 2023-01-24T18:59:04.367 INFO:tasks.workunit.client.0.smithi121.stdout: 9 1.00000 osd.9 up 1.00000 1.00000 2023-01-24T18:59:04.375 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph tell 'osd.*' injectargs --osd_max_markdown_count 1024 --osd_max_markdown_period 1 2023-01-24T18:59:04.505 INFO:tasks.workunit.client.0.smithi121.stdout:osd.0: {} 2023-01-24T18:59:04.536 INFO:tasks.workunit.client.0.smithi121.stdout:osd.1: {} 2023-01-24T18:59:04.567 INFO:tasks.workunit.client.0.smithi121.stdout:osd.2: {} 2023-01-24T18:59:04.599 INFO:tasks.workunit.client.0.smithi121.stdout:osd.3: {} 2023-01-24T18:59:04.631 INFO:tasks.workunit.client.0.smithi121.stdout:osd.4: {} 2023-01-24T18:59:04.661 INFO:tasks.workunit.client.0.smithi121.stdout:osd.5: {} 2023-01-24T18:59:04.693 INFO:tasks.workunit.client.0.smithi121.stdout:osd.6: {} 2023-01-24T18:59:04.724 INFO:tasks.workunit.client.0.smithi121.stdout:osd.7: {} 2023-01-24T18:59:04.756 INFO:tasks.workunit.client.0.smithi121.stdout:osd.8: {} 2023-01-24T18:59:04.788 INFO:tasks.workunit.client.0.smithi121.stdout:osd.9: {} 2023-01-24T18:59:04.798 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd set noout 2023-01-24T18:59:06.951 INFO:tasks.workunit.client.0.smithi121.stderr:noout is set 2023-01-24T18:59:06.963 INFO:tasks.workunit.client.0.smithi121.stderr:+ test_mark_two_osds_same_host_down 2023-01-24T18:59:06.963 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd set noup 2023-01-24T18:59:08.960 INFO:tasks.workunit.client.0.smithi121.stderr:noup is set 2023-01-24T18:59:08.971 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd down osd.0 osd.1 2023-01-24T18:59:09.266 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:09.264+0000 7f6d0212d700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-01-24T18:59:10.090 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:10.089+0000 7f6d0212d700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2023-01-24T18:59:10.093 INFO:tasks.workunit.client.0.smithi121.stderr:osd.0 is already down. osd.1 is already down. 2023-01-24T18:59:10.103 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:10.639 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:10.640 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-24T18:59:10.640 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-24T18:59:10.640 INFO:tasks.workunit.client.0.smithi121.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:10.640 INFO:tasks.workunit.client.0.smithi121.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:10.641 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-24T18:59:10.641 INFO:tasks.workunit.client.0.smithi121.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-24T18:59:10.651 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:10.652 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 host' 2023-01-24T18:59:11.202 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:11.203 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:11.203 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '2 osds' 2023-01-24T18:59:11.748 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:11.749 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:11.749 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.0 2023-01-24T18:59:11.808 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:11.806+0000 7fee566e5700 -1 osd.1 27 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:11.969 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:11.967+0000 7fee45cc8700 -1 osd.1 28 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:12.303 INFO:tasks.workunit.client.0.smithi121.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:12.304 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:12.304 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.1 2023-01-24T18:59:12.858 INFO:tasks.workunit.client.0.smithi121.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:12.859 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd unset noup 2023-01-24T18:59:12.973 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:12.971+0000 7fee45cc8700 -1 osd.1 29 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:13.445 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:13.444+0000 7fa31fbe5700 -1 osd.0 29 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:13.977 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:13.975+0000 7fa3171d8700 -1 osd.0 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:13.978 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:13.974+0000 7fee4e4d9700 -1 osd.1 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:14.981 INFO:tasks.workunit.client.0.smithi121.stderr:noup is unset 2023-01-24T18:59:14.992 INFO:tasks.workunit.client.0.smithi121.stderr:+ wait_for_healthy 2023-01-24T18:59:14.992 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:14.993 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep down 2023-01-24T18:59:15.547 INFO:tasks.workunit.client.0.smithi121.stderr:+ test_mark_two_osds_same_rack_down 2023-01-24T18:59:15.547 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd set noup 2023-01-24T18:59:16.991 INFO:tasks.workunit.client.0.smithi121.stderr:noup is set 2023-01-24T18:59:17.003 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd down osd.8 osd.9 2023-01-24T18:59:17.300 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:17.299+0000 7f6d0212d700 -1 mon.a@0(leader).osd e33 definitely_dead 0 2023-01-24T18:59:18.128 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:18.127+0000 7f6d0212d700 -1 mon.a@0(leader).osd e34 definitely_dead 0 2023-01-24T18:59:18.131 INFO:tasks.workunit.client.0.smithi121.stderr:osd.8 is already down. osd.9 is already down. 2023-01-24T18:59:18.140 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:18.673 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-24T18:59:18.674 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-24T18:59:18.674 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-24T18:59:18.674 INFO:tasks.workunit.client.0.smithi121.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-24T18:59:18.674 INFO:tasks.workunit.client.0.smithi121.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-24T18:59:18.674 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-24T18:59:18.675 INFO:tasks.workunit.client.0.smithi121.stdout: host host4 (root=root,row=row1,rack=rack2) (2 osds) is down 2023-01-24T18:59:18.675 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_RACK_DOWN: 1 rack (2 osds) down 2023-01-24T18:59:18.675 INFO:tasks.workunit.client.0.smithi121.stdout: rack rack2 (root=root,row=row1) (2 osds) is down 2023-01-24T18:59:18.675 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_ROW_DOWN: 1 row (2 osds) down 2023-01-24T18:59:18.675 INFO:tasks.workunit.client.0.smithi121.stdout: row row1 (root=root) (2 osds) is down 2023-01-24T18:59:18.685 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:18.685 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 host' 2023-01-24T18:59:19.231 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-24T18:59:19.231 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:19.232 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 rack' 2023-01-24T18:59:19.743 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:19.741+0000 7fa37687d700 -1 osd.8 34 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:19.794 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-24T18:59:19.794 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:19.794 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 row' 2023-01-24T18:59:20.004 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:20.002+0000 7fa365e60700 -1 osd.8 35 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:20.344 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-24T18:59:20.345 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:20.345 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '2 osds' 2023-01-24T18:59:20.386 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:59:20.385+0000 7fdfd8c19700 -1 osd.9 35 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:20.893 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-24T18:59:20.894 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:20.894 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.8 2023-01-24T18:59:21.440 INFO:tasks.workunit.client.0.smithi121.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-24T18:59:21.441 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:21.441 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.9 2023-01-24T18:59:21.984 INFO:tasks.workunit.client.0.smithi121.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-24T18:59:21.985 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd unset noup 2023-01-24T18:59:23.008 INFO:tasks.ceph.osd.9.smithi121.stderr:2023-01-24T18:59:23.007+0000 7fdfd0a0d700 -1 osd.9 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:23.009 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:23.006+0000 7fa36e671700 -1 osd.8 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:24.015 INFO:tasks.workunit.client.0.smithi121.stderr:noup is unset 2023-01-24T18:59:24.026 INFO:tasks.workunit.client.0.smithi121.stderr:+ wait_for_healthy 2023-01-24T18:59:24.027 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:24.027 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep down 2023-01-24T18:59:24.576 INFO:tasks.workunit.client.0.smithi121.stderr:+ test_mark_all_but_last_osds_down 2023-01-24T18:59:24.577 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd set noup 2023-01-24T18:59:26.024 INFO:tasks.workunit.client.0.smithi121.stderr:noup is set 2023-01-24T18:59:26.036 INFO:tasks.workunit.client.0.smithi121.stderr:++ ceph osd ls 2023-01-24T18:59:26.036 INFO:tasks.workunit.client.0.smithi121.stderr:++ sed '$d' 2023-01-24T18:59:26.451 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd down 0 1 2 3 4 5 6 7 8 2023-01-24T18:59:26.752 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:26.751+0000 7f6d0212d700 -1 mon.a@0(leader).osd e39 definitely_dead 0 2023-01-24T18:59:27.153 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:27.151+0000 7f6d0212d700 -1 mon.a@0(leader).osd e40 definitely_dead 0 2023-01-24T18:59:27.155 INFO:tasks.workunit.client.0.smithi121.stderr:osd.0 is already down. osd.1 is already down. osd.2 is already down. osd.3 is already down. osd.4 is already down. osd.5 is already down. osd.6 is already down. osd.7 is already down. osd.8 is already down. 2023-01-24T18:59:27.165 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:27.586 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:27.585+0000 7fa31e3e2700 -1 osd.0 40 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:27.685 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:27.683+0000 7fee54ee2700 -1 osd.1 40 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:27.709 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-24T18:59:27.710 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-24T18:59:27.710 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_DOWN: 9 osds down 2023-01-24T18:59:27.711 INFO:tasks.workunit.client.0.smithi121.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:27.711 INFO:tasks.workunit.client.0.smithi121.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:27.711 INFO:tasks.workunit.client.0.smithi121.stdout: osd.2 (root=root,row=row0,rack=rack0,host=host1) is down 2023-01-24T18:59:27.712 INFO:tasks.workunit.client.0.smithi121.stdout: osd.3 (root=root,row=row0,rack=rack0,host=host1) is down 2023-01-24T18:59:27.712 INFO:tasks.workunit.client.0.smithi121.stdout: osd.4 (root=root,row=row0,rack=rack1,host=host2) is down 2023-01-24T18:59:27.712 INFO:tasks.workunit.client.0.smithi121.stdout: osd.5 (root=root,row=row0,rack=rack1,host=host2) is down 2023-01-24T18:59:27.713 INFO:tasks.workunit.client.0.smithi121.stdout: osd.6 (root=root,row=row0,rack=rack1,host=host3) is down 2023-01-24T18:59:27.714 INFO:tasks.workunit.client.0.smithi121.stdout: osd.7 (root=root,row=row0,rack=rack1,host=host3) is down 2023-01-24T18:59:27.714 INFO:tasks.workunit.client.0.smithi121.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-24T18:59:27.714 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_HOST_DOWN: 4 hosts (8 osds) down 2023-01-24T18:59:27.714 INFO:tasks.workunit.client.0.smithi121.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-24T18:59:27.714 INFO:tasks.workunit.client.0.smithi121.stdout: host host1 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-24T18:59:27.715 INFO:tasks.workunit.client.0.smithi121.stdout: host host2 (root=root,row=row0,rack=rack1) (2 osds) is down 2023-01-24T18:59:27.715 INFO:tasks.workunit.client.0.smithi121.stdout: host host3 (root=root,row=row0,rack=rack1) (2 osds) is down 2023-01-24T18:59:27.715 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_RACK_DOWN: 2 racks (8 osds) down 2023-01-24T18:59:27.715 INFO:tasks.workunit.client.0.smithi121.stdout: rack rack0 (root=root,row=row0) (4 osds) is down 2023-01-24T18:59:27.715 INFO:tasks.workunit.client.0.smithi121.stdout: rack rack1 (root=root,row=row0) (4 osds) is down 2023-01-24T18:59:27.716 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_ROW_DOWN: 1 row (8 osds) down 2023-01-24T18:59:27.716 INFO:tasks.workunit.client.0.smithi121.stdout: row row0 (root=root) (8 osds) is down 2023-01-24T18:59:27.721 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:27.722 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 row' 2023-01-24T18:59:27.756 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:27.754+0000 7fa37507a700 -1 osd.8 40 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:27.817 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:59:27.815+0000 7f4f33249700 -1 osd.6 40 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.028 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:28.026+0000 7fa365e60700 -1 osd.8 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.028 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:28.026+0000 7fee45cc8700 -1 osd.1 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.029 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:59:28.026+0000 7f4f2282c700 -1 osd.6 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.029 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:28.027+0000 7fa30e9c7700 -1 osd.0 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.279 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-24T18:59:28.280 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:28.280 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '2 racks' 2023-01-24T18:59:28.524 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:59:28.522+0000 7fabb1b49700 -1 osd.4 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:28.830 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-24T18:59:28.831 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:28.831 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '4 hosts' 2023-01-24T18:59:29.032 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:59:29.030+0000 7f4f2282c700 -1 osd.6 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.033 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:29.030+0000 7fa365e60700 -1 osd.8 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.033 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:29.030+0000 7fee45cc8700 -1 osd.1 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.034 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:59:29.030+0000 7faba092b700 -1 osd.4 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.035 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:29.030+0000 7fa30e9c7700 -1 osd.0 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.154 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:59:29.153+0000 7ff36998e700 -1 osd.2 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.387 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-24T18:59:29.387 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:29.387 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '9 osds' 2023-01-24T18:59:29.493 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:59:29.492+0000 7fd4ddb9f700 -1 osd.5 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:29.948 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-24T18:59:29.949 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd unset noup 2023-01-24T18:59:30.033 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:59:30.031+0000 7ff358f71700 -1 osd.2 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.033 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:59:30.031+0000 7f4f2282c700 -1 osd.6 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.034 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:30.032+0000 7fee45cc8700 -1 osd.1 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.034 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:59:30.032+0000 7fd4cd182700 -1 osd.5 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.035 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:30.032+0000 7fa365e60700 -1 osd.8 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.035 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:59:30.032+0000 7faba092b700 -1 osd.4 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.035 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:30.032+0000 7fa30e9c7700 -1 osd.0 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:30.291 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:59:30.290+0000 7f86a23ee700 -1 osd.7 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:31.039 INFO:tasks.ceph.osd.7.smithi121.stderr:2023-01-24T18:59:31.037+0000 7f869a1e2700 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.040 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:31.037+0000 7fa3171d8700 -1 osd.0 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.040 INFO:tasks.ceph.osd.4.smithi121.stderr:2023-01-24T18:59:31.037+0000 7faba913c700 -1 osd.4 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.040 INFO:tasks.ceph.osd.8.smithi121.stderr:2023-01-24T18:59:31.038+0000 7fa36e671700 -1 osd.8 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.041 INFO:tasks.ceph.osd.2.smithi121.stderr:2023-01-24T18:59:31.037+0000 7ff361782700 -1 osd.2 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.041 INFO:tasks.ceph.osd.6.smithi121.stderr:2023-01-24T18:59:31.040+0000 7f4f2b03d700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.042 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:31.038+0000 7fee4e4d9700 -1 osd.1 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:31.042 INFO:tasks.ceph.osd.5.smithi121.stderr:2023-01-24T18:59:31.038+0000 7fd4d5993700 -1 osd.5 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:32.069 INFO:tasks.workunit.client.0.smithi121.stderr:noup is unset 2023-01-24T18:59:32.080 INFO:tasks.workunit.client.0.smithi121.stderr:+ wait_for_healthy 2023-01-24T18:59:32.081 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:32.081 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep down 2023-01-24T18:59:32.170 INFO:tasks.ceph.osd.3.smithi121.stderr:2023-01-24T18:59:32.169+0000 7f07c1529700 -1 osd.3 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:32.641 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noout flag(s) set; 1 osds down 2023-01-24T18:59:32.641 INFO:tasks.workunit.client.0.smithi121.stderr:+ sleep 1 2023-01-24T18:59:33.642 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:33.643 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep down 2023-01-24T18:59:34.204 INFO:tasks.workunit.client.0.smithi121.stderr:+ test_mark_two_osds_same_host_down_with_classes 2023-01-24T18:59:34.204 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd set noup 2023-01-24T18:59:36.073 INFO:tasks.workunit.client.0.smithi121.stderr:noup is set 2023-01-24T18:59:36.084 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd crush set-device-class ssd osd.0 osd.2 osd.4 osd.6 osd.8 2023-01-24T18:59:38.079 INFO:tasks.workunit.client.0.smithi121.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.2 already set to class ssd. set-device-class item id 2 name 'osd.2' device_class 'ssd': no change. osd.4 already set to class ssd. set-device-class item id 4 name 'osd.4' device_class 'ssd': no change. osd.6 already set to class ssd. set-device-class item id 6 name 'osd.6' device_class 'ssd': no change. osd.8 already set to class ssd. set-device-class item id 8 name 'osd.8' device_class 'ssd': no change. set osd(s) to class 'ssd' 2023-01-24T18:59:38.091 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd crush set-device-class hdd osd.1 osd.3 osd.5 osd.7 osd.9 2023-01-24T18:59:40.093 INFO:tasks.workunit.client.0.smithi121.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. osd.3 already set to class hdd. set-device-class item id 3 name 'osd.3' device_class 'hdd': no change. osd.5 already set to class hdd. set-device-class item id 5 name 'osd.5' device_class 'hdd': no change. osd.7 already set to class hdd. set-device-class item id 7 name 'osd.7' device_class 'hdd': no change. osd.9 already set to class hdd. set-device-class item id 9 name 'osd.9' device_class 'hdd': no change. set osd(s) to class 'hdd' 2023-01-24T18:59:40.104 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd down osd.0 osd.1 2023-01-24T18:59:40.404 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:40.401+0000 7f6d0212d700 -1 mon.a@0(leader).osd e53 definitely_dead 0 2023-01-24T18:59:41.220 INFO:tasks.ceph.mon.a.smithi121.stderr:2023-01-24T18:59:41.219+0000 7f6d0212d700 -1 mon.a@0(leader).osd e54 definitely_dead 0 2023-01-24T18:59:41.223 INFO:tasks.workunit.client.0.smithi121.stderr:osd.0 is already down. osd.1 is already down. 2023-01-24T18:59:41.233 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:41.644 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:41.643+0000 7fa31fbe5700 -1 osd.0 54 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:41.776 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:41.776 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-24T18:59:41.776 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-24T18:59:41.776 INFO:tasks.workunit.client.0.smithi121.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:41.777 INFO:tasks.workunit.client.0.smithi121.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:41.777 INFO:tasks.workunit.client.0.smithi121.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-24T18:59:41.777 INFO:tasks.workunit.client.0.smithi121.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-24T18:59:41.788 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:41.788 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '1 host' 2023-01-24T18:59:42.095 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:42.093+0000 7fa30e9c7700 -1 osd.0 55 osdmap NOUP flag is set, waiting for it to clear 2023-01-24T18:59:42.339 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:42.340 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:42.340 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep '2 osds' 2023-01-24T18:59:42.888 INFO:tasks.workunit.client.0.smithi121.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-24T18:59:42.889 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:42.889 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.0 2023-01-24T18:59:43.436 INFO:tasks.workunit.client.0.smithi121.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:43.437 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health detail 2023-01-24T18:59:43.437 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep osd.1 2023-01-24T18:59:43.987 INFO:tasks.workunit.client.0.smithi121.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-24T18:59:43.987 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph osd unset noup 2023-01-24T18:59:45.102 INFO:tasks.ceph.osd.0.smithi121.stderr:2023-01-24T18:59:45.101+0000 7fa3171d8700 -1 osd.0 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:45.672 INFO:tasks.ceph.osd.1.smithi121.stderr:2023-01-24T18:59:45.670+0000 7fee4e4d9700 -1 osd.1 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-24T18:59:46.109 INFO:tasks.workunit.client.0.smithi121.stderr:noup is unset 2023-01-24T18:59:46.120 INFO:tasks.workunit.client.0.smithi121.stderr:+ wait_for_healthy 2023-01-24T18:59:46.121 INFO:tasks.workunit.client.0.smithi121.stderr:+ ceph health 2023-01-24T18:59:46.121 INFO:tasks.workunit.client.0.smithi121.stderr:+ grep down 2023-01-24T18:59:46.683 INFO:tasks.workunit.client.0.smithi121.stderr:+ exit 0 2023-01-24T18:59:46.683 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-24T18:59:46.684 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-24T18:59:46.751 INFO:tasks.workunit:Stopping ['rados/test_health_warnings.sh'] on client.0... 2023-01-24T18:59:46.751 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-01-24T18:59:47.031 DEBUG:teuthology.parallel:result is None 2023-01-24T18:59:47.032 DEBUG:teuthology.orchestra.run.smithi121:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-01-24T18:59:47.056 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-01-24T18:59:47.057 DEBUG:teuthology.orchestra.run.smithi121:> rmdir -- /home/ubuntu/cephtest/mnt.0 2023-01-24T18:59:47.113 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2023-01-24T18:59:47.114 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-01-24T18:59:47.125 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-01-24T18:59:47.126 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-24T18:59:47.461 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:47.461 INFO:teuthology.orchestra.run.smithi121.stdout:{"epoch":58,"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","created":"2023-01-24T18:57:31.792588+0000","modified":"2023-01-24T18:59:47.104499+0000","last_up_change":"2023-01-24T18:59:46.100201+0000","last_in_change":"2023-01-24T18:57:34.908758+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-24T18:57:44.174050+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"15954bbe-06b0-4485-9410-769c1c3cb37d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":56,"up_from":57,"up_thru":34,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6808","nonce":86474},{"type":"v1","addr":"172.21.15.121:6809","nonce":86474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6802","nonce":3086474},{"type":"v1","addr":"172.21.15.121:6803","nonce":3086474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6814","nonce":86474},{"type":"v1","addr":"172.21.15.121:6815","nonce":86474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6812","nonce":86474},{"type":"v1","addr":"172.21.15.121:6813","nonce":86474}]},"public_addr":"172.21.15.121:6809/86474","cluster_addr":"172.21.15.121:6803/3086474","heartbeat_back_addr":"172.21.15.121:6815/86474","heartbeat_front_addr":"172.21.15.121:6813/86474","state":["exists","up"]},{"osd":1,"uuid":"8327917a-b7e7-4b48-a86b-f964974cb9d8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":56,"up_from":57,"up_thru":57,"down_at":54,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6840","nonce":86476},{"type":"v1","addr":"172.21.15.121:6842","nonce":86476}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6882","nonce":3086476},{"type":"v1","addr":"172.21.15.121:6883","nonce":3086476}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6857","nonce":86476},{"type":"v1","addr":"172.21.15.121:6861","nonce":86476}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6848","nonce":86476},{"type":"v1","addr":"172.21.15.121:6853","nonce":86476}]},"public_addr":"172.21.15.121:6842/86476","cluster_addr":"172.21.15.121:6883/3086476","heartbeat_back_addr":"172.21.15.121:6861/86476","heartbeat_front_addr":"172.21.15.121:6853/86476","state":["exists","up"]},{"osd":2,"uuid":"7f00e786-6387-4398-acd5-3b99bb1d11c5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":0,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6841","nonce":86481},{"type":"v1","addr":"172.21.15.121:6843","nonce":86481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6818","nonce":1086481},{"type":"v1","addr":"172.21.15.121:6819","nonce":1086481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6855","nonce":86481},{"type":"v1","addr":"172.21.15.121:6859","nonce":86481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6849","nonce":86481},{"type":"v1","addr":"172.21.15.121:6852","nonce":86481}]},"public_addr":"172.21.15.121:6843/86481","cluster_addr":"172.21.15.121:6819/1086481","heartbeat_back_addr":"172.21.15.121:6859/86481","heartbeat_front_addr":"172.21.15.121:6852/86481","state":["exists","up"]},{"osd":3,"uuid":"ba02d479-684f-4d8e-85c1-bec43226002b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":45,"up_from":46,"up_thru":57,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6800","nonce":86478},{"type":"v1","addr":"172.21.15.121:6801","nonce":86478}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6829","nonce":1086478},{"type":"v1","addr":"172.21.15.121:6831","nonce":1086478}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6806","nonce":86478},{"type":"v1","addr":"172.21.15.121:6807","nonce":86478}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6804","nonce":86478},{"type":"v1","addr":"172.21.15.121:6805","nonce":86478}]},"public_addr":"172.21.15.121:6801/86478","cluster_addr":"172.21.15.121:6831/1086478","heartbeat_back_addr":"172.21.15.121:6807/86478","heartbeat_front_addr":"172.21.15.121:6805/86478","state":["exists","up"]},{"osd":4,"uuid":"dc808a7f-78de-4825-9229-ec4f148b83c0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":57,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6816","nonce":86515},{"type":"v1","addr":"172.21.15.121:6817","nonce":86515}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6828","nonce":1086515},{"type":"v1","addr":"172.21.15.121:6830","nonce":1086515}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6822","nonce":86515},{"type":"v1","addr":"172.21.15.121:6823","nonce":86515}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6820","nonce":86515},{"type":"v1","addr":"172.21.15.121:6821","nonce":86515}]},"public_addr":"172.21.15.121:6817/86515","cluster_addr":"172.21.15.121:6830/1086515","heartbeat_back_addr":"172.21.15.121:6823/86515","heartbeat_front_addr":"172.21.15.121:6821/86515","state":["exists","up"]},{"osd":5,"uuid":"a468112f-5f12-4014-a343-3d202bd83fb9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":45,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6851","nonce":86522},{"type":"v1","addr":"172.21.15.121:6856","nonce":86522}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6845","nonce":1086522},{"type":"v1","addr":"172.21.15.121:6847","nonce":1086522}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6869","nonce":86522},{"type":"v1","addr":"172.21.15.121:6871","nonce":86522}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6865","nonce":86522},{"type":"v1","addr":"172.21.15.121:6867","nonce":86522}]},"public_addr":"172.21.15.121:6856/86522","cluster_addr":"172.21.15.121:6847/1086522","heartbeat_back_addr":"172.21.15.121:6871/86522","heartbeat_front_addr":"172.21.15.121:6867/86522","state":["exists","up"]},{"osd":6,"uuid":"f38bbe03-38e6-427e-97fe-a73f27fbae78","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":54,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6824","nonce":86480},{"type":"v1","addr":"172.21.15.121:6826","nonce":86480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6844","nonce":1086480},{"type":"v1","addr":"172.21.15.121:6846","nonce":1086480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6836","nonce":86480},{"type":"v1","addr":"172.21.15.121:6839","nonce":86480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6832","nonce":86480},{"type":"v1","addr":"172.21.15.121:6834","nonce":86480}]},"public_addr":"172.21.15.121:6826/86480","cluster_addr":"172.21.15.121:6846/1086480","heartbeat_back_addr":"172.21.15.121:6839/86480","heartbeat_front_addr":"172.21.15.121:6834/86480","state":["exists","up"]},{"osd":7,"uuid":"ffe4c402-2d51-44f3-b9fd-69191fa98596","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":46,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6825","nonce":86494},{"type":"v1","addr":"172.21.15.121:6827","nonce":86494}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6860","nonce":1086494},{"type":"v1","addr":"172.21.15.121:6863","nonce":1086494}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6837","nonce":86494},{"type":"v1","addr":"172.21.15.121:6838","nonce":86494}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6833","nonce":86494},{"type":"v1","addr":"172.21.15.121:6835","nonce":86494}]},"public_addr":"172.21.15.121:6827/86494","cluster_addr":"172.21.15.121:6863/1086494","heartbeat_back_addr":"172.21.15.121:6838/86494","heartbeat_front_addr":"172.21.15.121:6835/86494","state":["exists","up"]},{"osd":8,"uuid":"d4940a67-8d4c-4416-bc6a-4cfcc1e7853d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":17,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6850","nonce":86523},{"type":"v1","addr":"172.21.15.121:6854","nonce":86523}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6874","nonce":2086523},{"type":"v1","addr":"172.21.15.121:6875","nonce":2086523}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6868","nonce":86523},{"type":"v1","addr":"172.21.15.121:6870","nonce":86523}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6864","nonce":86523},{"type":"v1","addr":"172.21.15.121:6866","nonce":86523}]},"public_addr":"172.21.15.121:6854/86523","cluster_addr":"172.21.15.121:6875/2086523","heartbeat_back_addr":"172.21.15.121:6870/86523","heartbeat_front_addr":"172.21.15.121:6866/86523","state":["exists","up"]},{"osd":9,"uuid":"db696b96-e8bf-4e4f-938a-d6c6d4e2d000","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":36,"up_from":37,"up_thru":57,"down_at":34,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6872","nonce":86631},{"type":"v1","addr":"172.21.15.121:6873","nonce":86631}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6858","nonce":1086631},{"type":"v1","addr":"172.21.15.121:6862","nonce":1086631}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6878","nonce":86631},{"type":"v1","addr":"172.21.15.121:6879","nonce":86631}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.121:6876","nonce":86631},{"type":"v1","addr":"172.21.15.121:6877","nonce":86631}]},"public_addr":"172.21.15.121:6873/86631","cluster_addr":"172.21.15.121:6862/1086631","heartbeat_back_addr":"172.21.15.121:6879/86631","heartbeat_front_addr":"172.21.15.121:6877/86631","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2023-01-24T18:59:41.081828+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.167472+0000","dead_epoch":54},{"osd":1,"down_stamp":"2023-01-24T18:59:41.081828+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388192+0000","dead_epoch":55},{"osd":2,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.417277+0000","dead_epoch":41},{"osd":3,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.172991+0000","dead_epoch":44},{"osd":4,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.178997+0000","dead_epoch":41},{"osd":5,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.379661+0000","dead_epoch":42},{"osd":6,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.388286+0000","dead_epoch":40},{"osd":7,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.419300+0000","dead_epoch":42},{"osd":8,"down_stamp":"2023-01-24T18:59:27.011954+0000","laggy_probability":0.50999999046325684,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.407306+0000","dead_epoch":40},{"osd":9,"down_stamp":"2023-01-24T18:59:17.991038+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-01-24T18:57:39.459694+0000","dead_epoch":34}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-24T18:59:47.474 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-24T18:59:47.475 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:59:47.773 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:47.775 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:59:47.786 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-01-24T18:59:47.388488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58268,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":1048517732,"statfs":{"total":1073741824000,"available":1073682157568,"internally_reserved":0,"allocated":6303744,"data_stored":2976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-01-24T18:59:42.097012+0000","last_change":"2023-01-24T18:59:34.766165+0000","last_active":"2023-01-24T18:59:42.097012+0000","last_peered":"2023-01-24T18:59:42.097012+0000","last_clean":"2023-01-24T18:59:42.097012+0000","last_became_active":"2023-01-24T18:59:34.765899+0000","last_became_peered":"2023-01-24T18:59:34.765899+0000","last_unstale":"2023-01-24T18:59:42.097012+0000","last_undegraded":"2023-01-24T18:59:42.097012+0000","last_fullsized":"2023-01-24T18:59:42.097012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:20:29.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742676+0000","last_change":"2023-01-24T18:59:33.056035+0000","last_active":"2023-01-24T18:59:41.742676+0000","last_peered":"2023-01-24T18:59:41.742676+0000","last_clean":"2023-01-24T18:59:41.742676+0000","last_became_active":"2023-01-24T18:59:33.055784+0000","last_became_peered":"2023-01-24T18:59:33.055784+0000","last_unstale":"2023-01-24T18:59:41.742676+0000","last_undegraded":"2023-01-24T18:59:41.742676+0000","last_fullsized":"2023-01-24T18:59:41.742676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T02:50:57.279433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.743239+0000","last_change":"2023-01-24T18:59:34.061386+0000","last_active":"2023-01-24T18:59:41.743239+0000","last_peered":"2023-01-24T18:59:41.743239+0000","last_clean":"2023-01-24T18:59:41.743239+0000","last_became_active":"2023-01-24T18:59:34.061168+0000","last_became_peered":"2023-01-24T18:59:34.061168+0000","last_unstale":"2023-01-24T18:59:41.743239+0000","last_undegraded":"2023-01-24T18:59:41.743239+0000","last_fullsized":"2023-01-24T18:59:41.743239+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:47:08.393138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742186+0000","last_change":"2023-01-24T18:59:33.055365+0000","last_active":"2023-01-24T18:59:41.742186+0000","last_peered":"2023-01-24T18:59:41.742186+0000","last_clean":"2023-01-24T18:59:41.742186+0000","last_became_active":"2023-01-24T18:59:33.055107+0000","last_became_peered":"2023-01-24T18:59:33.055107+0000","last_unstale":"2023-01-24T18:59:41.742186+0000","last_undegraded":"2023-01-24T18:59:41.742186+0000","last_fullsized":"2023-01-24T18:59:41.742186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:28:20.693149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.098015+0000","last_change":"2023-01-24T18:59:42.097939+0000","last_active":"2023-01-24T18:59:42.098015+0000","last_peered":"2023-01-24T18:59:42.098015+0000","last_clean":"2023-01-24T18:59:40.213562+0000","last_became_active":"2023-01-24T18:59:42.097939+0000","last_became_peered":"2023-01-24T18:59:42.097939+0000","last_unstale":"2023-01-24T18:59:42.098015+0000","last_undegraded":"2023-01-24T18:59:42.098015+0000","last_fullsized":"2023-01-24T18:59:42.097363+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:14:00.773569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":56,"state":"active+undersized","last_fresh":"2023-01-24T18:59:45.101684+0000","last_change":"2023-01-24T18:59:45.101684+0000","last_active":"2023-01-24T18:59:45.101684+0000","last_peered":"2023-01-24T18:59:45.101684+0000","last_clean":"2023-01-24T18:59:40.213001+0000","last_became_active":"2023-01-24T18:59:42.097662+0000","last_became_peered":"2023-01-24T18:59:42.097662+0000","last_unstale":"2023-01-24T18:59:45.101684+0000","last_undegraded":"2023-01-24T18:59:45.101684+0000","last_fullsized":"2023-01-24T18:59:42.097020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:00:32.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.097783+0000","last_change":"2023-01-24T18:59:42.097699+0000","last_active":"2023-01-24T18:59:42.097783+0000","last_peered":"2023-01-24T18:59:42.097783+0000","last_clean":"2023-01-24T18:59:27.307673+0000","last_became_active":"2023-01-24T18:59:42.097699+0000","last_became_peered":"2023-01-24T18:59:42.097699+0000","last_unstale":"2023-01-24T18:59:42.097783+0000","last_undegraded":"2023-01-24T18:59:42.097783+0000","last_fullsized":"2023-01-24T18:59:42.097086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:44:51.146459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+undersized","last_fresh":"2023-01-24T18:59:42.095543+0000","last_change":"2023-01-24T18:59:42.095510+0000","last_active":"2023-01-24T18:59:42.095543+0000","last_peered":"2023-01-24T18:59:42.095543+0000","last_clean":"2023-01-24T18:59:40.214096+0000","last_became_active":"2023-01-24T18:59:42.095510+0000","last_became_peered":"2023-01-24T18:59:42.095510+0000","last_unstale":"2023-01-24T18:59:42.095543+0000","last_undegraded":"2023-01-24T18:59:42.095543+0000","last_fullsized":"2023-01-24T18:59:42.094804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:45:35.964857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.374}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.437},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.39200000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.315},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.36499999999999999}]},{"osd":7,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.42299999999999999}]}]},{"osd":7,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.501}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.51700000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48999999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.746},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.37},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40500000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41799999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.435}]}]},{"osd":1,"up_from":0,"seq":27,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5880,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851720,"statfs":{"total":107374182400,"available":107368161280,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.312}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27600000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.51000000000000001}]}]},{"osd":0,"up_from":0,"seq":27,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.312}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.4329999999999998,"5min":8.4329999999999998,"15min":8.4329999999999998},"last":0.253}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.4470000000000001}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.1719999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":7.5039999999999996,"5min":7.5039999999999996,"15min":7.5039999999999996},"last":1.244},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.268}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.1950000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":1.26}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48599999999999999}]}]},{"osd":2,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5740,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851860,"statfs":{"total":107374182400,"available":107368304640,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.25},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.307}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.46200000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.50700000000000001}]}]},{"osd":3,"up_from":46,"seq":197568495643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.1070000000000002,"5min":4.1070000000000002,"15min":4.1070000000000002},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.499},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.39100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.379}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.438},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":45,"seq":193273528347,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":5.7549999999999999,"5min":5.7549999999999999,"15min":5.7549999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.502}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.5830000000000002,"5min":5.5830000000000002,"15min":5.5830000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.3920000000000003,"5min":5.3920000000000003,"15min":5.3920000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.38},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":5.3490000000000002,"5min":5.3490000000000002,"15min":5.3490000000000002},"last":0.42199999999999999}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":4.6379999999999999,"5min":4.6379999999999999,"15min":4.6379999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.7850000000000001,"5min":5.7850000000000001,"15min":5.7850000000000001},"last":0.69199999999999995}]}]},{"osd":5,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.503},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.441}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.378},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.314},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.46800000000000003}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53200000000000003}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:59:47.789 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:59:48.087 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:48.089 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:59:48.100 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-01-24T18:59:47.388488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58268,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":1048517732,"statfs":{"total":1073741824000,"available":1073682157568,"internally_reserved":0,"allocated":6303744,"data_stored":2976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-01-24T18:59:42.097012+0000","last_change":"2023-01-24T18:59:34.766165+0000","last_active":"2023-01-24T18:59:42.097012+0000","last_peered":"2023-01-24T18:59:42.097012+0000","last_clean":"2023-01-24T18:59:42.097012+0000","last_became_active":"2023-01-24T18:59:34.765899+0000","last_became_peered":"2023-01-24T18:59:34.765899+0000","last_unstale":"2023-01-24T18:59:42.097012+0000","last_undegraded":"2023-01-24T18:59:42.097012+0000","last_fullsized":"2023-01-24T18:59:42.097012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:20:29.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742676+0000","last_change":"2023-01-24T18:59:33.056035+0000","last_active":"2023-01-24T18:59:41.742676+0000","last_peered":"2023-01-24T18:59:41.742676+0000","last_clean":"2023-01-24T18:59:41.742676+0000","last_became_active":"2023-01-24T18:59:33.055784+0000","last_became_peered":"2023-01-24T18:59:33.055784+0000","last_unstale":"2023-01-24T18:59:41.742676+0000","last_undegraded":"2023-01-24T18:59:41.742676+0000","last_fullsized":"2023-01-24T18:59:41.742676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T02:50:57.279433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.743239+0000","last_change":"2023-01-24T18:59:34.061386+0000","last_active":"2023-01-24T18:59:41.743239+0000","last_peered":"2023-01-24T18:59:41.743239+0000","last_clean":"2023-01-24T18:59:41.743239+0000","last_became_active":"2023-01-24T18:59:34.061168+0000","last_became_peered":"2023-01-24T18:59:34.061168+0000","last_unstale":"2023-01-24T18:59:41.743239+0000","last_undegraded":"2023-01-24T18:59:41.743239+0000","last_fullsized":"2023-01-24T18:59:41.743239+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:47:08.393138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742186+0000","last_change":"2023-01-24T18:59:33.055365+0000","last_active":"2023-01-24T18:59:41.742186+0000","last_peered":"2023-01-24T18:59:41.742186+0000","last_clean":"2023-01-24T18:59:41.742186+0000","last_became_active":"2023-01-24T18:59:33.055107+0000","last_became_peered":"2023-01-24T18:59:33.055107+0000","last_unstale":"2023-01-24T18:59:41.742186+0000","last_undegraded":"2023-01-24T18:59:41.742186+0000","last_fullsized":"2023-01-24T18:59:41.742186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:28:20.693149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.098015+0000","last_change":"2023-01-24T18:59:42.097939+0000","last_active":"2023-01-24T18:59:42.098015+0000","last_peered":"2023-01-24T18:59:42.098015+0000","last_clean":"2023-01-24T18:59:40.213562+0000","last_became_active":"2023-01-24T18:59:42.097939+0000","last_became_peered":"2023-01-24T18:59:42.097939+0000","last_unstale":"2023-01-24T18:59:42.098015+0000","last_undegraded":"2023-01-24T18:59:42.098015+0000","last_fullsized":"2023-01-24T18:59:42.097363+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:14:00.773569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":56,"state":"active+undersized","last_fresh":"2023-01-24T18:59:45.101684+0000","last_change":"2023-01-24T18:59:45.101684+0000","last_active":"2023-01-24T18:59:45.101684+0000","last_peered":"2023-01-24T18:59:45.101684+0000","last_clean":"2023-01-24T18:59:40.213001+0000","last_became_active":"2023-01-24T18:59:42.097662+0000","last_became_peered":"2023-01-24T18:59:42.097662+0000","last_unstale":"2023-01-24T18:59:45.101684+0000","last_undegraded":"2023-01-24T18:59:45.101684+0000","last_fullsized":"2023-01-24T18:59:42.097020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:00:32.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.097783+0000","last_change":"2023-01-24T18:59:42.097699+0000","last_active":"2023-01-24T18:59:42.097783+0000","last_peered":"2023-01-24T18:59:42.097783+0000","last_clean":"2023-01-24T18:59:27.307673+0000","last_became_active":"2023-01-24T18:59:42.097699+0000","last_became_peered":"2023-01-24T18:59:42.097699+0000","last_unstale":"2023-01-24T18:59:42.097783+0000","last_undegraded":"2023-01-24T18:59:42.097783+0000","last_fullsized":"2023-01-24T18:59:42.097086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:44:51.146459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+undersized","last_fresh":"2023-01-24T18:59:42.095543+0000","last_change":"2023-01-24T18:59:42.095510+0000","last_active":"2023-01-24T18:59:42.095543+0000","last_peered":"2023-01-24T18:59:42.095543+0000","last_clean":"2023-01-24T18:59:40.214096+0000","last_became_active":"2023-01-24T18:59:42.095510+0000","last_became_peered":"2023-01-24T18:59:42.095510+0000","last_unstale":"2023-01-24T18:59:42.095543+0000","last_undegraded":"2023-01-24T18:59:42.095543+0000","last_fullsized":"2023-01-24T18:59:42.094804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:45:35.964857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.374}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.437},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.39200000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.315},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.36499999999999999}]},{"osd":7,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.42299999999999999}]}]},{"osd":7,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.501}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.51700000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48999999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.746},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.37},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40500000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41799999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.435}]}]},{"osd":1,"up_from":0,"seq":27,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5880,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851720,"statfs":{"total":107374182400,"available":107368161280,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.312}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27600000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.51000000000000001}]}]},{"osd":0,"up_from":0,"seq":27,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.312}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.4329999999999998,"5min":8.4329999999999998,"15min":8.4329999999999998},"last":0.253}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.4470000000000001}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.1719999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":7.5039999999999996,"5min":7.5039999999999996,"15min":7.5039999999999996},"last":1.244},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.268}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.1950000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":1.26}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48599999999999999}]}]},{"osd":2,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5740,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851860,"statfs":{"total":107374182400,"available":107368304640,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.25},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.307}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.46200000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.50700000000000001}]}]},{"osd":3,"up_from":46,"seq":197568495643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.1070000000000002,"5min":4.1070000000000002,"15min":4.1070000000000002},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.499},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.39100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.379}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.438},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":45,"seq":193273528347,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":5.7549999999999999,"5min":5.7549999999999999,"15min":5.7549999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.502}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.5830000000000002,"5min":5.5830000000000002,"15min":5.5830000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.3920000000000003,"5min":5.3920000000000003,"15min":5.3920000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.38},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":5.3490000000000002,"5min":5.3490000000000002,"15min":5.3490000000000002},"last":0.42199999999999999}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":4.6379999999999999,"5min":4.6379999999999999,"15min":4.6379999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.7850000000000001,"5min":5.7850000000000001,"15min":5.7850000000000001},"last":0.69199999999999995}]}]},{"osd":5,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.503},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.441}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.378},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.314},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.46800000000000003}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53200000000000003}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:59:48.103 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:59:48.407 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:48.409 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:59:48.419 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-01-24T18:59:47.388488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58268,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":1048517732,"statfs":{"total":1073741824000,"available":1073682157568,"internally_reserved":0,"allocated":6303744,"data_stored":2976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-01-24T18:59:42.097012+0000","last_change":"2023-01-24T18:59:34.766165+0000","last_active":"2023-01-24T18:59:42.097012+0000","last_peered":"2023-01-24T18:59:42.097012+0000","last_clean":"2023-01-24T18:59:42.097012+0000","last_became_active":"2023-01-24T18:59:34.765899+0000","last_became_peered":"2023-01-24T18:59:34.765899+0000","last_unstale":"2023-01-24T18:59:42.097012+0000","last_undegraded":"2023-01-24T18:59:42.097012+0000","last_fullsized":"2023-01-24T18:59:42.097012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:20:29.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742676+0000","last_change":"2023-01-24T18:59:33.056035+0000","last_active":"2023-01-24T18:59:41.742676+0000","last_peered":"2023-01-24T18:59:41.742676+0000","last_clean":"2023-01-24T18:59:41.742676+0000","last_became_active":"2023-01-24T18:59:33.055784+0000","last_became_peered":"2023-01-24T18:59:33.055784+0000","last_unstale":"2023-01-24T18:59:41.742676+0000","last_undegraded":"2023-01-24T18:59:41.742676+0000","last_fullsized":"2023-01-24T18:59:41.742676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T02:50:57.279433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.743239+0000","last_change":"2023-01-24T18:59:34.061386+0000","last_active":"2023-01-24T18:59:41.743239+0000","last_peered":"2023-01-24T18:59:41.743239+0000","last_clean":"2023-01-24T18:59:41.743239+0000","last_became_active":"2023-01-24T18:59:34.061168+0000","last_became_peered":"2023-01-24T18:59:34.061168+0000","last_unstale":"2023-01-24T18:59:41.743239+0000","last_undegraded":"2023-01-24T18:59:41.743239+0000","last_fullsized":"2023-01-24T18:59:41.743239+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:47:08.393138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742186+0000","last_change":"2023-01-24T18:59:33.055365+0000","last_active":"2023-01-24T18:59:41.742186+0000","last_peered":"2023-01-24T18:59:41.742186+0000","last_clean":"2023-01-24T18:59:41.742186+0000","last_became_active":"2023-01-24T18:59:33.055107+0000","last_became_peered":"2023-01-24T18:59:33.055107+0000","last_unstale":"2023-01-24T18:59:41.742186+0000","last_undegraded":"2023-01-24T18:59:41.742186+0000","last_fullsized":"2023-01-24T18:59:41.742186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:28:20.693149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.098015+0000","last_change":"2023-01-24T18:59:42.097939+0000","last_active":"2023-01-24T18:59:42.098015+0000","last_peered":"2023-01-24T18:59:42.098015+0000","last_clean":"2023-01-24T18:59:40.213562+0000","last_became_active":"2023-01-24T18:59:42.097939+0000","last_became_peered":"2023-01-24T18:59:42.097939+0000","last_unstale":"2023-01-24T18:59:42.098015+0000","last_undegraded":"2023-01-24T18:59:42.098015+0000","last_fullsized":"2023-01-24T18:59:42.097363+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:14:00.773569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":56,"state":"active+undersized","last_fresh":"2023-01-24T18:59:45.101684+0000","last_change":"2023-01-24T18:59:45.101684+0000","last_active":"2023-01-24T18:59:45.101684+0000","last_peered":"2023-01-24T18:59:45.101684+0000","last_clean":"2023-01-24T18:59:40.213001+0000","last_became_active":"2023-01-24T18:59:42.097662+0000","last_became_peered":"2023-01-24T18:59:42.097662+0000","last_unstale":"2023-01-24T18:59:45.101684+0000","last_undegraded":"2023-01-24T18:59:45.101684+0000","last_fullsized":"2023-01-24T18:59:42.097020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:00:32.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.097783+0000","last_change":"2023-01-24T18:59:42.097699+0000","last_active":"2023-01-24T18:59:42.097783+0000","last_peered":"2023-01-24T18:59:42.097783+0000","last_clean":"2023-01-24T18:59:27.307673+0000","last_became_active":"2023-01-24T18:59:42.097699+0000","last_became_peered":"2023-01-24T18:59:42.097699+0000","last_unstale":"2023-01-24T18:59:42.097783+0000","last_undegraded":"2023-01-24T18:59:42.097783+0000","last_fullsized":"2023-01-24T18:59:42.097086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:44:51.146459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+undersized","last_fresh":"2023-01-24T18:59:42.095543+0000","last_change":"2023-01-24T18:59:42.095510+0000","last_active":"2023-01-24T18:59:42.095543+0000","last_peered":"2023-01-24T18:59:42.095543+0000","last_clean":"2023-01-24T18:59:40.214096+0000","last_became_active":"2023-01-24T18:59:42.095510+0000","last_became_peered":"2023-01-24T18:59:42.095510+0000","last_unstale":"2023-01-24T18:59:42.095543+0000","last_undegraded":"2023-01-24T18:59:42.095543+0000","last_fullsized":"2023-01-24T18:59:42.094804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:45:35.964857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.374}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.437},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.39200000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.315},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.36499999999999999}]},{"osd":7,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.42299999999999999}]}]},{"osd":7,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.501}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.51700000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48999999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.746},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.37},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40500000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41799999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.435}]}]},{"osd":1,"up_from":0,"seq":27,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5880,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851720,"statfs":{"total":107374182400,"available":107368161280,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.312}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27600000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.51000000000000001}]}]},{"osd":0,"up_from":0,"seq":27,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.312}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.4329999999999998,"5min":8.4329999999999998,"15min":8.4329999999999998},"last":0.253}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.4470000000000001}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.1719999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":7.5039999999999996,"5min":7.5039999999999996,"15min":7.5039999999999996},"last":1.244},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.268}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.1950000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":1.26}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48599999999999999}]}]},{"osd":2,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5740,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851860,"statfs":{"total":107374182400,"available":107368304640,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.25},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.307}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.46200000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.50700000000000001}]}]},{"osd":3,"up_from":46,"seq":197568495643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.1070000000000002,"5min":4.1070000000000002,"15min":4.1070000000000002},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.499},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.39100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.379}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.438},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":45,"seq":193273528347,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":5.7549999999999999,"5min":5.7549999999999999,"15min":5.7549999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.502}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.5830000000000002,"5min":5.5830000000000002,"15min":5.5830000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.3920000000000003,"5min":5.3920000000000003,"15min":5.3920000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.38},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":5.3490000000000002,"5min":5.3490000000000002,"15min":5.3490000000000002},"last":0.42199999999999999}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":4.6379999999999999,"5min":4.6379999999999999,"15min":4.6379999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.7850000000000001,"5min":5.7850000000000001,"15min":5.7850000000000001},"last":0.69199999999999995}]}]},{"osd":5,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.503},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.441}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.378},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.314},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.46800000000000003}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53200000000000003}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:59:48.422 INFO:tasks.ceph.ceph_manager.ceph:PG 1.2 is not active+clean 2023-01-24T18:59:48.422 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.2', 'version': "0'0", 'reported_seq': 66, 'reported_epoch': 55, 'state': 'active+undersized+wait', 'last_fresh': '2023-01-24T18:59:42.098015+0000', 'last_change': '2023-01-24T18:59:42.097939+0000', 'last_active': '2023-01-24T18:59:42.098015+0000', 'last_peered': '2023-01-24T18:59:42.098015+0000', 'last_clean': '2023-01-24T18:59:40.213562+0000', 'last_became_active': '2023-01-24T18:59:42.097939+0000', 'last_became_peered': '2023-01-24T18:59:42.097939+0000', 'last_unstale': '2023-01-24T18:59:42.098015+0000', 'last_undegraded': '2023-01-24T18:59:42.098015+0000', 'last_fullsized': '2023-01-24T18:59:42.097363+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_clean_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-25T21:14:00.773569+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4], 'acting': [4], 'avail_no_missing': ['4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-01-24T18:59:48.423 INFO:tasks.ceph.ceph_manager.ceph:PG 1.3 is not active+clean 2023-01-24T18:59:48.423 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.3', 'version': "0'0", 'reported_seq': 67, 'reported_epoch': 56, 'state': 'active+undersized', 'last_fresh': '2023-01-24T18:59:45.101684+0000', 'last_change': '2023-01-24T18:59:45.101684+0000', 'last_active': '2023-01-24T18:59:45.101684+0000', 'last_peered': '2023-01-24T18:59:45.101684+0000', 'last_clean': '2023-01-24T18:59:40.213001+0000', 'last_became_active': '2023-01-24T18:59:42.097662+0000', 'last_became_peered': '2023-01-24T18:59:42.097662+0000', 'last_unstale': '2023-01-24T18:59:45.101684+0000', 'last_undegraded': '2023-01-24T18:59:45.101684+0000', 'last_fullsized': '2023-01-24T18:59:42.097020+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 47, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_clean_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T06:00:32.567064+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3], 'acting': [3], 'avail_no_missing': ['3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-01-24T18:59:48.423 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2023-01-24T18:59:48.424 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 43, 'reported_epoch': 55, 'state': 'active+undersized+wait', 'last_fresh': '2023-01-24T18:59:42.097783+0000', 'last_change': '2023-01-24T18:59:42.097699+0000', 'last_active': '2023-01-24T18:59:42.097783+0000', 'last_peered': '2023-01-24T18:59:42.097783+0000', 'last_clean': '2023-01-24T18:59:27.307673+0000', 'last_became_active': '2023-01-24T18:59:42.097699+0000', 'last_became_peered': '2023-01-24T18:59:42.097699+0000', 'last_unstale': '2023-01-24T18:59:42.097783+0000', 'last_undegraded': '2023-01-24T18:59:42.097783+0000', 'last_fullsized': '2023-01-24T18:59:42.097086+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_clean_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T00:44:51.146459+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6], 'acting': [6], 'avail_no_missing': ['6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-01-24T18:59:48.424 INFO:tasks.ceph.ceph_manager.ceph:PG 1.5 is not active+clean 2023-01-24T18:59:48.424 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.5', 'version': "0'0", 'reported_seq': 83, 'reported_epoch': 55, 'state': 'active+undersized', 'last_fresh': '2023-01-24T18:59:42.095543+0000', 'last_change': '2023-01-24T18:59:42.095510+0000', 'last_active': '2023-01-24T18:59:42.095543+0000', 'last_peered': '2023-01-24T18:59:42.095543+0000', 'last_clean': '2023-01-24T18:59:40.214096+0000', 'last_became_active': '2023-01-24T18:59:42.095510+0000', 'last_became_peered': '2023-01-24T18:59:42.095510+0000', 'last_unstale': '2023-01-24T18:59:42.095543+0000', 'last_undegraded': '2023-01-24T18:59:42.095543+0000', 'last_fullsized': '2023-01-24T18:59:42.094804+0000', 'mapping_epoch': 54, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'last_clean_scrub_stamp': '2023-01-24T18:57:44.855510+0000', 'objects_scrubbed': 0, 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T04:45:35.964857+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9], 'acting': [9], 'avail_no_missing': ['9'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-01-24T18:59:48.425 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-01-24T18:59:48.798 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:48.799 INFO:teuthology.orchestra.run.smithi121.stdout:{"fsid":"b27849be-aeb9-4bd2-adcf-d31dbae5a00d","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":137,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":1},"osdmap":{"epoch":58,"num_osds":10,"num_up_osds":10,"osd_up_since":1674586786,"num_in_osds":10,"osd_in_since":1674586654,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":4},{"state_name":"active+undersized","count":2},{"state_name":"active+undersized+wait","count":2}],"num_pgs":8,"num_pools":1,"num_objects":0,"data_bytes":0,"bytes_used":59666432,"bytes_avail":1073682157568,"bytes_total":1073741824000},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-01-24T18:58:41.377643+0000","services":{}},"progress_events":{}} 2023-01-24T18:59:48.811 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-01-24T18:59:48.812 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:59:49.112 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:49.114 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:59:49.125 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":113,"stamp":"2023-01-24T18:59:47.388488+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58268,"kb_used_data":6156,"kb_used_omap":0,"kb_used_meta":52032,"kb_avail":1048517732,"statfs":{"total":1073741824000,"available":1073682157568,"internally_reserved":0,"allocated":6303744,"data_stored":2976722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53280768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":55,"state":"active+clean","last_fresh":"2023-01-24T18:59:42.097012+0000","last_change":"2023-01-24T18:59:34.766165+0000","last_active":"2023-01-24T18:59:42.097012+0000","last_peered":"2023-01-24T18:59:42.097012+0000","last_clean":"2023-01-24T18:59:42.097012+0000","last_became_active":"2023-01-24T18:59:34.765899+0000","last_became_peered":"2023-01-24T18:59:34.765899+0000","last_unstale":"2023-01-24T18:59:42.097012+0000","last_undegraded":"2023-01-24T18:59:42.097012+0000","last_fullsized":"2023-01-24T18:59:42.097012+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:20:29.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":56,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742676+0000","last_change":"2023-01-24T18:59:33.056035+0000","last_active":"2023-01-24T18:59:41.742676+0000","last_peered":"2023-01-24T18:59:41.742676+0000","last_clean":"2023-01-24T18:59:41.742676+0000","last_became_active":"2023-01-24T18:59:33.055784+0000","last_became_peered":"2023-01-24T18:59:33.055784+0000","last_unstale":"2023-01-24T18:59:41.742676+0000","last_undegraded":"2023-01-24T18:59:41.742676+0000","last_fullsized":"2023-01-24T18:59:41.742676+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T02:50:57.279433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":47,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.743239+0000","last_change":"2023-01-24T18:59:34.061386+0000","last_active":"2023-01-24T18:59:41.743239+0000","last_peered":"2023-01-24T18:59:41.743239+0000","last_clean":"2023-01-24T18:59:41.743239+0000","last_became_active":"2023-01-24T18:59:34.061168+0000","last_became_peered":"2023-01-24T18:59:34.061168+0000","last_unstale":"2023-01-24T18:59:41.743239+0000","last_undegraded":"2023-01-24T18:59:41.743239+0000","last_fullsized":"2023-01-24T18:59:41.743239+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:47:08.393138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-01-24T18:59:41.742186+0000","last_change":"2023-01-24T18:59:33.055365+0000","last_active":"2023-01-24T18:59:41.742186+0000","last_peered":"2023-01-24T18:59:41.742186+0000","last_clean":"2023-01-24T18:59:41.742186+0000","last_became_active":"2023-01-24T18:59:33.055107+0000","last_became_peered":"2023-01-24T18:59:33.055107+0000","last_unstale":"2023-01-24T18:59:41.742186+0000","last_undegraded":"2023-01-24T18:59:41.742186+0000","last_fullsized":"2023-01-24T18:59:41.742186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:28:20.693149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":66,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.098015+0000","last_change":"2023-01-24T18:59:42.097939+0000","last_active":"2023-01-24T18:59:42.098015+0000","last_peered":"2023-01-24T18:59:42.098015+0000","last_clean":"2023-01-24T18:59:40.213562+0000","last_became_active":"2023-01-24T18:59:42.097939+0000","last_became_peered":"2023-01-24T18:59:42.097939+0000","last_unstale":"2023-01-24T18:59:42.098015+0000","last_undegraded":"2023-01-24T18:59:42.098015+0000","last_fullsized":"2023-01-24T18:59:42.097363+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T21:14:00.773569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":67,"reported_epoch":56,"state":"active+undersized","last_fresh":"2023-01-24T18:59:45.101684+0000","last_change":"2023-01-24T18:59:45.101684+0000","last_active":"2023-01-24T18:59:45.101684+0000","last_peered":"2023-01-24T18:59:45.101684+0000","last_clean":"2023-01-24T18:59:40.213001+0000","last_became_active":"2023-01-24T18:59:42.097662+0000","last_became_peered":"2023-01-24T18:59:42.097662+0000","last_unstale":"2023-01-24T18:59:45.101684+0000","last_undegraded":"2023-01-24T18:59:45.101684+0000","last_fullsized":"2023-01-24T18:59:42.097020+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:00:32.567064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":43,"reported_epoch":55,"state":"active+undersized+wait","last_fresh":"2023-01-24T18:59:42.097783+0000","last_change":"2023-01-24T18:59:42.097699+0000","last_active":"2023-01-24T18:59:42.097783+0000","last_peered":"2023-01-24T18:59:42.097783+0000","last_clean":"2023-01-24T18:59:27.307673+0000","last_became_active":"2023-01-24T18:59:42.097699+0000","last_became_peered":"2023-01-24T18:59:42.097699+0000","last_unstale":"2023-01-24T18:59:42.097783+0000","last_undegraded":"2023-01-24T18:59:42.097783+0000","last_fullsized":"2023-01-24T18:59:42.097086+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T00:44:51.146459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6],"acting":[6],"avail_no_missing":["6"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":83,"reported_epoch":55,"state":"active+undersized","last_fresh":"2023-01-24T18:59:42.095543+0000","last_change":"2023-01-24T18:59:42.095510+0000","last_active":"2023-01-24T18:59:42.095543+0000","last_peered":"2023-01-24T18:59:42.095543+0000","last_clean":"2023-01-24T18:59:40.214096+0000","last_became_active":"2023-01-24T18:59:42.095510+0000","last_became_peered":"2023-01-24T18:59:42.095510+0000","last_unstale":"2023-01-24T18:59:42.095543+0000","last_undegraded":"2023-01-24T18:59:42.095543+0000","last_fullsized":"2023-01-24T18:59:42.094804+0000","mapping_epoch":54,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:45:35.964857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":12,"acting":12,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789980,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.374}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.308}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.39900000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.52100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.41999999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.437},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.46100000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.39200000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.371},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.24399999999999999}]},{"osd":3,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.315},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.36499999999999999}]},{"osd":7,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.42299999999999999}]}]},{"osd":7,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.57899999999999996}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.438},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.501}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.56899999999999995}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.51700000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.48999999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.58599999999999997}]}]},{"osd":6,"up_from":45,"seq":193273528347,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.78500000000000003}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.746},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.45600000000000002}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.37},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.47099999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.49199999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.50600000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.40500000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.41799999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.435}]}]},{"osd":1,"up_from":0,"seq":27,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5880,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851720,"statfs":{"total":107374182400,"available":107368161280,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.42099999999999999}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.22600000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.312}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.248},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.27600000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.51500000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.52400000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.49299999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.503},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.51000000000000001}]}]},{"osd":0,"up_from":0,"seq":27,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.312}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.4329999999999998,"5min":8.4329999999999998,"15min":8.4329999999999998},"last":0.253}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":1.4470000000000001}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":1.0640000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":1.0429999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":1.2250000000000001},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":1.1719999999999999}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":7.5039999999999996,"5min":7.5039999999999996,"15min":7.5039999999999996},"last":1.244},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":1.268}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":1.1950000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":1.26}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.48599999999999999}]}]},{"osd":2,"up_from":45,"seq":193273528347,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5740,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851860,"statfs":{"total":107374182400,"available":107368304640,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.25},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.36899999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38400000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.307}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.46200000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.436}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.50700000000000001}]}]},{"osd":3,"up_from":46,"seq":197568495643,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5816,"kb_used_data":624,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851784,"statfs":{"total":107374182400,"available":107368226816,"internally_reserved":0,"allocated":638976,"data_stored":303044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":0.84399999999999997}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.60199999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.29999999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.44500000000000001}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.1070000000000002,"5min":4.1070000000000002,"15min":4.1070000000000002},"last":0.34899999999999998}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.499},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.39100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.379}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.438},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.56000000000000005}]}]},{"osd":4,"up_from":45,"seq":193273528347,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5868,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851732,"statfs":{"total":107374182400,"available":107368173568,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.433},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":5.7549999999999999,"5min":5.7549999999999999,"15min":5.7549999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.502}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.5830000000000002,"5min":5.5830000000000002,"15min":5.5830000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.3920000000000003,"5min":5.3920000000000003,"15min":5.3920000000000003},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.38},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":0.45200000000000001}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.35399999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":5.3490000000000002,"5min":5.3490000000000002,"15min":5.3490000000000002},"last":0.42199999999999999}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":4.6379999999999999,"5min":4.6379999999999999,"15min":4.6379999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.67600000000000005}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":5.3719999999999999,"5min":5.3719999999999999,"15min":5.3719999999999999},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":5.7850000000000001,"5min":5.7850000000000001,"15min":5.7850000000000001},"last":0.69199999999999995}]}]},{"osd":5,"up_from":45,"seq":193273528347,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":295370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"last":0.503},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.495,"5min":0.495,"15min":0.495},"last":0.441}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.378},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"last":0.53700000000000003}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.42799999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.314},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.46800000000000003}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.45000000000000001}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.38100000000000001}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.53200000000000003}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-24T18:59:52.128 DEBUG:teuthology.orchestra.run.smithi121:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-24T18:59:52.441 INFO:teuthology.orchestra.run.smithi121.stdout: 2023-01-24T18:59:52.443 INFO:teuthology.orchestra.run.smithi121.stderr:dumped all 2023-01-24T18:59:52.454 INFO:teuthology.orchestra.run.smithi121.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2023-01-24T18:59:51.389100+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58784,"kb_used_data":6480,"kb_used_omap":0,"kb_used_meta":52224,"kb_avail":1048517216,"statfs":{"total":1073741824000,"available":1073681629184,"internally_reserved":0,"allocated":6635520,"data_stored":3191640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53477376},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":46,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-24T18:59:46.622660+0000","last_change":"2023-01-24T18:59:34.766165+0000","last_active":"2023-01-24T18:59:46.622660+0000","last_peered":"2023-01-24T18:59:46.622660+0000","last_clean":"2023-01-24T18:59:46.622660+0000","last_became_active":"2023-01-24T18:59:34.765899+0000","last_became_peered":"2023-01-24T18:59:34.765899+0000","last_unstale":"2023-01-24T18:59:46.622660+0000","last_undegraded":"2023-01-24T18:59:46.622660+0000","last_fullsized":"2023-01-24T18:59:46.622660+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:20:29.400241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":58,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-24T18:59:46.623112+0000","last_change":"2023-01-24T18:59:33.056035+0000","last_active":"2023-01-24T18:59:46.623112+0000","last_peered":"2023-01-24T18:59:46.623112+0000","last_clean":"2023-01-24T18:59:46.623112+0000","last_became_active":"2023-01-24T18:59:33.055784+0000","last_became_peered":"2023-01-24T18:59:33.055784+0000","last_unstale":"2023-01-24T18:59:46.623112+0000","last_undegraded":"2023-01-24T18:59:46.623112+0000","last_fullsized":"2023-01-24T18:59:46.623112+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T02:50:57.279433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":49,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-01-24T18:59:46.530984+0000","last_change":"2023-01-24T18:59:34.061386+0000","last_active":"2023-01-24T18:59:46.530984+0000","last_peered":"2023-01-24T18:59:46.530984+0000","last_clean":"2023-01-24T18:59:46.530984+0000","last_became_active":"2023-01-24T18:59:34.061168+0000","last_became_peered":"2023-01-24T18:59:34.061168+0000","last_unstale":"2023-01-24T18:59:46.530984+0000","last_undegraded":"2023-01-24T18:59:46.530984+0000","last_fullsized":"2023-01-24T18:59:46.530984+0000","mapping_epoch":46,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":47,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:47:08.393138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":64,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-01-24T18:59:48.142667+0000","last_change":"2023-01-24T18:59:33.055365+0000","last_active":"2023-01-24T18:59:48.142667+0000","last_peered":"2023-01-24T18:59:48.142667+0000","last_clean":"2023-01-24T18:59:48.142667+0000","last_became_active":"2023-01-24T18:59:33.055107+0000","last_became_peered":"2023-01-24T18:59:33.055107+0000","last_unstale":"2023-01-24T18:59:48.142667+0000","last_undegraded":"2023-01-24T18:59:48.142667+0000","last_fullsized":"2023-01-24T18:59:48.142667+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-25T19:28:20.693149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":79,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-24T18:59:47.116713+0000","last_change":"2023-01-24T18:59:47.116713+0000","last_active":"2023-01-24T18:59:47.116713+0000","last_peered":"2023-01-24T18:59:47.116713+0000","last_clean":"2023-01-24T18:59:47.116713+0000","last_became_active":"2023-01-24T18:59:47.116433+0000","last_became_peered":"2023-01-24T18:59:47.116433+0000","last_unstale":"2023-01-24T18:59:47.116713+0000","last_undegraded":"2023-01-24T18:59:47.116713+0000","last_fullsized":"2023-01-24T18:59:47.116713+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T03:31:08.762532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":78,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-24T18:59:47.116573+0000","last_change":"2023-01-24T18:59:47.116573+0000","last_active":"2023-01-24T18:59:47.116573+0000","last_peered":"2023-01-24T18:59:47.116573+0000","last_clean":"2023-01-24T18:59:47.116573+0000","last_became_active":"2023-01-24T18:59:47.116342+0000","last_became_peered":"2023-01-24T18:59:47.116342+0000","last_unstale":"2023-01-24T18:59:47.116573+0000","last_undegraded":"2023-01-24T18:59:47.116573+0000","last_fullsized":"2023-01-24T18:59:47.116573+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T03:24:49.503796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-24T18:59:47.758911+0000","last_change":"2023-01-24T18:59:47.758911+0000","last_active":"2023-01-24T18:59:47.758911+0000","last_peered":"2023-01-24T18:59:47.758911+0000","last_clean":"2023-01-24T18:59:47.758911+0000","last_became_active":"2023-01-24T18:59:47.758662+0000","last_became_peered":"2023-01-24T18:59:47.758662+0000","last_unstale":"2023-01-24T18:59:47.758911+0000","last_undegraded":"2023-01-24T18:59:47.758911+0000","last_fullsized":"2023-01-24T18:59:47.758911+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:34:51.761567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-24T18:59:47.321184+0000","last_change":"2023-01-24T18:59:47.321184+0000","last_active":"2023-01-24T18:59:47.321184+0000","last_peered":"2023-01-24T18:59:47.321184+0000","last_clean":"2023-01-24T18:59:47.321184+0000","last_became_active":"2023-01-24T18:59:47.320933+0000","last_became_peered":"2023-01-24T18:59:47.320933+0000","last_unstale":"2023-01-24T18:59:47.321184+0000","last_undegraded":"2023-01-24T18:59:47.321184+0000","last_fullsized":"2023-01-24T18:59:47.321184+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-24T18:57:44.855510+0000","last_clean_scrub_stamp":"2023-01-24T18:57:44.855510+0000","objects_scrubbed":0,"log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T01:03:45.948934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789981,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"min":{"1min":0.19600000000000001,"5min":0.19600000000000001,"15min":0.19600000000000001},"max":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.374}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.1699999999999999,"5min":1.1699999999999999,"15min":1.1699999999999999},"last":0.34300000000000003}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.433,"15min":0.433},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.216},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.36099999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"last":0.318}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.46200000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.373}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.44900000000000001}]}]},{"osd":8,"up_from":45,"seq":193273528348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.36699999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"last":0.442},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.84399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.86499999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":0.441}]},{"osd":3,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.496},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.61499999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005}]},{"osd":5,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.5840000000000001,"5min":1.5840000000000001,"15min":1.5840000000000001},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.23,"5min":1.23,"15min":1.23},"last":0.50800000000000001}]},{"osd":9,"last update":"Tue Jan 24 18:58:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.611,"5min":1.611,"15min":1.611},"last":0.63400000000000001}]}]},{"osd":7,"up_from":45,"seq":193273528348,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.44},{"interface":"front","average":{"1min":0.379,"5min":0.379,"15min":0.379},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"last":0.42299999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"last":0.35999999999999999}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.436,"5min":0.436,"15min":0.436},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.498}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.624},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.052,"5min":1.052,"15min":1.052},"last":0.66100000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.5129999999999999,"5min":1.5129999999999999,"15min":1.5129999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.60099999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.63400000000000001}]}]},{"osd":6,"up_from":45,"seq":193273528348,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.432,"5min":0.432,"15min":0.432},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.51100000000000001}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.628}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.86499999999999999}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.91900000000000004}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.94699999999999995}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.091,"5min":1.091,"15min":1.091},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.76800000000000002}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.121,"5min":1.121,"15min":1.121},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.74199999999999999}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.90200000000000002}]}]},{"osd":1,"up_from":57,"seq":244813135900,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"min":{"1min":0.245,"5min":0.245,"15min":0.245},"max":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.26200000000000001}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"last":0.38800000000000001}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.427,"5min":1.427,"15min":1.427},"last":0.317}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.55200000000000005}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":1.4710000000000001,"5min":1.4710000000000001,"15min":1.4710000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.621}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.7529999999999999,"5min":1.7529999999999999,"15min":1.7529999999999999},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.61199999999999999}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.56399999999999995}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.99,"5min":1.99,"15min":1.99},"last":0.69799999999999995}]}]},{"osd":0,"up_from":57,"seq":244813135900,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":8.4329999999999998,"5min":8.4329999999999998,"15min":8.4329999999999998},"last":0.23799999999999999}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.308,"5min":0.308,"15min":0.308},"max":{"1min":1.7350000000000001,"5min":1.7350000000000001,"15min":1.7350000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.56899999999999995}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":1.1080000000000001,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.81599999999999995,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64500000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":7.5039999999999996,"5min":7.5039999999999996,"15min":7.5039999999999996},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2649999999999999,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.89600000000000002}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.768,"5min":1.768,"15min":1.768},"last":0.63}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.98099999999999998}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.71699999999999997}]}]},{"osd":2,"up_from":45,"seq":193273528348,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.373,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.33200000000000002}]},{"osd":1,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.629,"5min":0.629,"15min":0.629},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.33200000000000002}]},{"osd":4,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.35999999999999999}]},{"osd":5,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.39400000000000002}]},{"osd":6,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":1.073},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":1.127,"5min":1.127,"15min":1.127},"last":0.60999999999999999}]},{"osd":7,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.175,"5min":1.175,"15min":1.175},"last":0.505}]},{"osd":8,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"last":0.51900000000000002}]},{"osd":9,"last update":"Tue Jan 24 18:58:41 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.55500000000000005}]}]},{"osd":3,"up_from":46,"seq":197568495644,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":2.6499999999999999,"5min":2.6499999999999999,"15min":2.6499999999999999},"last":0.27900000000000003}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":3.6139999999999999,"5min":3.6139999999999999,"15min":3.6139999999999999},"last":0.252}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.308},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":2.4849999999999999,"5min":2.4849999999999999,"15min":2.4849999999999999},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.373},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":2.6829999999999998,"5min":2.6829999999999998,"15min":2.6829999999999998},"last":0.33000000000000002}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":4.1070000000000002,"5min":4.1070000000000002,"15min":4.1070000000000002},"last":0.41699999999999998}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":3.5089999999999999,"5min":3.5089999999999999,"15min":3.5089999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":4.1760000000000002,"5min":4.1760000000000002,"15min":4.1760000000000002},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.626},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":2.71,"5min":2.71,"15min":2.71},"last":0.40500000000000003}]},{"osd":8,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.339,"5min":1.339,"15min":1.339},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":2.6040000000000001,"5min":2.6040000000000001,"15min":2.6040000000000001},"last":0.501}]},{"osd":9,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.835,"5min":1.835,"15min":1.835},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":2.5649999999999999,"5min":2.5649999999999999,"15min":2.5649999999999999},"last":0.64300000000000002}]}]},{"osd":4,"up_from":45,"seq":193273528348,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":319164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.2670000000000003,"5min":5.2670000000000003,"15min":5.2670000000000003},"last":0.60899999999999999}]},{"osd":1,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":5.7549999999999999,"5min":5.7549999999999999,"15min":5.7549999999999999},"last":0.502},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":5.3010000000000002,"5min":5.3010000000000002,"15min":5.3010000000000002},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.24299999999999999},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.44900000000000001}]},{"osd":3,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":5.5830000000000002,"5min":5.5830000000000002,"15min":5.5830000000000002},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":5.3920000000000003,"5min":5.3920000000000003,"15min":5.3920000000000003},"last":0.73299999999999998}]},{"osd":5,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":5.617,"5min":5.617,"15min":5.617},"last":0.63600000000000001}]},{"osd":6,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":5.3250000000000002,"5min":5.3250000000000002,"15min":5.3250000000000002},"last":0.626},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.746}]},{"osd":7,"last update":"Tue Jan 24 18:58:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.1519999999999999,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":5.3490000000000002,"5min":5.3490000000000002,"15min":5.3490000000000002},"last":0.80300000000000005}]},{"osd":8