2023-01-25T04:08:06.957 INFO:root:teuthology version: 0.0.1.dev22+g8e1bba1 2023-01-25T04:08:06.958 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --name yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi --description rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} --archive /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136725 --owner scheduled_yuriw@teuthology -- /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136725/orig.config.yaml 2023-01-25T04:08:06.989 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2023-01-25T04:08:07.060 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136725 branch: wip-yuri-testing-2023-01-23-0926 description: rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}} email: yweinstein@radhat.com first_in_suite: false job_id: '7136725' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuri-testing-2023-01-23-0926 ceph: conf: global: mon election default strategy: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bluestore zero block detection: true debug ms: 1 debug osd: 20 osd debug verify cached snaps: true osd debug verify missing on start: true osd mclock override recovery settings: true osd op queue: debug_random osd op queue cut off: debug_random flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: b0d2592529629ab4f86fed5991e556a87dcda563 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 install: ceph: flavor: default sha1: b0d2592529629ab4f86fed5991e556a87dcda563 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: wip-yuri-testing-2023-01-23-0926 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 owner: scheduled_yuriw@teuthology priority: 71 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - osd.3 - osd.4 - osd.5 - osd.6 - osd.7 - osd.8 - osd.9 - client.0 sha1: b0d2592529629ab4f86fed5991e556a87dcda563 sleep_before_teardown: 0 suite: rados suite_branch: wip-yuri-testing-2023-01-23-0926 suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: b0d2592529629ab4f86fed5991e556a87dcda563 targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: null - ceph: conf: osd: osd max object name len: 400 osd max object namespace len: 64 log-ignorelist: - but it is still running - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ pre-mgr-commands: - sudo ceph config set mgr mgr_pool false --force - workunit: clients: all: - rados/test_health_warnings.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8e1bba12e584b7ae912e92e94afe9c075ab51fa3 timestamp: 2023-01-24_22:20:59 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.17735 2023-01-25T04:08:07.061 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa; will attempt to use it 2023-01-25T04:08:07.061 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa/tasks 2023-01-25T04:08:07.062 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-01-25T04:08:07.063 INFO:teuthology.task.internal:Checking packages... 2023-01-25T04:08:07.095 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash 'b0d2592529629ab4f86fed5991e556a87dcda563' 2023-01-25T04:08:07.095 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-01-25T04:08:07.096 INFO:teuthology.packaging:ref: None 2023-01-25T04:08:07.096 INFO:teuthology.packaging:tag: None 2023-01-25T04:08:07.096 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-01-23-0926 2023-01-25T04:08:07.096 INFO:teuthology.packaging:sha1: b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:08:07.097 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-01-23-0926 2023-01-25T04:08:07.254 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-1959.gb0d25925 2023-01-25T04:08:07.255 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-01-25T04:08:07.265 INFO:teuthology.task.internal:no buildpackages task found 2023-01-25T04:08:07.265 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-01-25T04:08:07.348 INFO:teuthology.task.internal:Saving configuration 2023-01-25T04:08:07.359 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-01-25T04:08:07.389 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-01-25T04:08:07.415 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136725', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-01-25 04:03:31.932665', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-01-25T04:08:07.415 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-01-25T04:08:07.422 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0'] 2023-01-25T04:08:07.423 INFO:teuthology.run_tasks:Running task console_log... 2023-01-25T04:08:07.474 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f90449d25e0>, signals=[15]) 2023-01-25T04:08:07.475 INFO:teuthology.run_tasks:Running task internal.connect... 2023-01-25T04:08:07.482 INFO:teuthology.task.internal:Opening connections... 2023-01-25T04:08:07.482 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:08:07.484 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:08:07.552 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-01-25T04:08:07.559 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2023-01-25T04:08:07.580 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2023-01-25T04:08:07.580 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2023-01-25T04:08:07.640 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="Red Hat Enterprise Linux" 2023-01-25T04:08:07.640 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="8.6 (Ootpa)" 2023-01-25T04:08:07.640 INFO:teuthology.orchestra.run.smithi016.stdout:ID="rhel" 2023-01-25T04:08:07.641 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="fedora" 2023-01-25T04:08:07.641 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="8.6" 2023-01-25T04:08:07.641 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el8" 2023-01-25T04:08:07.641 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-01-25T04:08:07.642 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2023-01-25T04:08:07.642 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-01-25T04:08:07.642 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://www.redhat.com/" 2023-01-25T04:08:07.642 INFO:teuthology.orchestra.run.smithi016.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-01-25T04:08:07.642 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-01-25T04:08:07.643 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:08:07.643 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-01-25T04:08:07.643 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-01-25T04:08:07.643 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-01-25T04:08:07.643 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-01-25T04:08:07.644 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2023-01-25T04:08:07.661 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-01-25T04:08:07.673 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-01-25T04:08:07.710 INFO:teuthology.task.internal:Checking for old test directory... 2023-01-25T04:08:07.711 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2023-01-25T04:08:07.730 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-01-25T04:08:07.737 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-01-25T04:08:07.737 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2023-01-25T04:08:07.811 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-01-25T04:08:07.845 INFO:teuthology.run_tasks:Running task kernel... 2023-01-25T04:08:07.898 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-01-25T04:08:07.899 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-01-25T04:08:07.899 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2023-01-25T04:08:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:08:07.920 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2023-01-25T04:08:07.980 INFO:teuthology.orchestra.run.smithi016.stdout:4.18.0-372.9.1.el8.x86_64 2023-01-25T04:08:07.981 INFO:teuthology.task.kernel:Running kernel on smithi016: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:08:07.981 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2023-01-25T04:08:09.195 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:08:09.195 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T04:08:10.728 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2023-01-25T04:08:11.662 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 15 MB/s | 13 MB 00:00 2023-01-25T04:08:14.879 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 441 kB/s | 24 kB 00:00 2023-01-25T04:08:15.578 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-01-25T04:08:15.638 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:08:15.639 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:08:15.639 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:08:15.771 DEBUG:teuthology.orchestra.run.smithi016:> echo no | sudo yum reinstall kernel || true 2023-01-25T04:08:16.138 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:08:16.139 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T04:08:16.407 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Jan 2023 04:08:14 AM UTC. 2023-01-25T04:08:16.461 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:08:16.493 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-01-25T04:08:16.540 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum reinstall -y kernel || true 2023-01-25T04:08:16.894 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:08:16.894 INFO:teuthology.orchestra.run.smithi016.stdout:Unable to read consumer identity 2023-01-25T04:08:17.165 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Jan 2023 04:08:14 AM UTC. 2023-01-25T04:08:17.211 INFO:teuthology.orchestra.run.smithi016.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-01-25T04:08:17.246 INFO:teuthology.orchestra.run.smithi016.stderr:Error: No packages marked for reinstall. 2023-01-25T04:08:17.293 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel --last | head -n 1 2023-01-25T04:08:17.376 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-4.18.0-372.9.1.el8.x86_64 Thu 14 Jul 2022 06:23:21 PM UTC 2023-01-25T04:08:17.378 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-01-25T04:08:17.378 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-01-25T04:08:17.378 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-01-25T04:08:17.379 DEBUG:teuthology.orchestra.run.smithi016:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-01-25T04:08:17.441 INFO:teuthology.orchestra.run.smithi016.stdout:ttyS1 2023-01-25T04:08:17.456 DEBUG:teuthology.parallel:result is None 2023-01-25T04:08:17.456 INFO:teuthology.run_tasks:Running task internal.base... 2023-01-25T04:08:17.468 INFO:teuthology.task.internal:Creating test directory... 2023-01-25T04:08:17.469 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-01-25T04:08:17.486 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-01-25T04:08:17.496 INFO:teuthology.run_tasks:Running task internal.archive... 2023-01-25T04:08:17.565 INFO:teuthology.task.internal:Creating archive directory... 2023-01-25T04:08:17.566 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-01-25T04:08:17.589 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-01-25T04:08:17.602 INFO:teuthology.task.internal:Enabling coredump saving... 2023-01-25T04:08:17.603 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-01-25T04:08:17.669 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:08:17.680 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-01-25T04:08:17.682 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-01-25T04:08:17.696 INFO:teuthology.task.internal:Configuring sudo... 2023-01-25T04:08:17.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-01-25T04:08:17.726 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-01-25T04:08:17.749 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-01-25T04:08:17.750 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-01-25T04:08:17.784 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:08:17.869 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-01-25T04:08:17.943 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:08:17.997 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-01-25T04:08:18.029 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:08:18.030 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-01-25T04:08:18.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2023-01-25T04:08:18.187 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-01-25T04:08:18.559 INFO:teuthology.run_tasks:Running task internal.timer... 2023-01-25T04:08:18.570 INFO:teuthology.task.internal:Starting timer... 2023-01-25T04:08:18.570 INFO:teuthology.run_tasks:Running task pcp... 2023-01-25T04:08:18.697 INFO:teuthology.run_tasks:Running task selinux... 2023-01-25T04:08:18.723 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-01-25T04:08:18.724 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2023-01-25T04:08:18.779 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2023-01-25T04:08:18.781 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-01-25T04:08:18.781 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2023-01-25T04:08:18.808 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2023-01-25T04:08:18.808 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive'} 2023-01-25T04:08:18.809 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|comm="sss_cache"\|context=system_u:system_r:NetworkManager_dispatcher_t:s0\)' 2023-01-25T04:08:18.875 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:08:18.877 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-01-25T04:08:18.877 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2023-01-25T04:08:18.945 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-01-25T04:08:18.956 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-01-25T04:08:18.956 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-01-25T04:08:18.974 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-01-25T04:08:18.976 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-01-25T04:15:16.588 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com')] 2023-01-25T04:15:16.589 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2023-01-25T04:15:16.590 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-01-25T04:15:16.660 DEBUG:teuthology.orchestra.run.smithi016:> true 2023-01-25T04:15:16.730 INFO:teuthology.run_tasks:Running task clock... 2023-01-25T04:15:16.741 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-01-25T04:15:16.742 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-01-25T04:15:16.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-01-25T04:15:16.804 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-01-25T04:15:16.819 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-01-25T04:15:16.889 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2023-01-25T04:15:16.901 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2023-01-25T04:15:16.916 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-01-25T04:15:16.930 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-01-25T04:15:17.013 INFO:teuthology.orchestra.run.smithi016.stderr:bash: ntpq: command not found 2023-01-25T04:15:17.016 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-01-25T04:15:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2023-01-25T04:15:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:15:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:15:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:15:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-01-25T04:15:17.018 INFO:teuthology.run_tasks:Running task install... 2023-01-25T04:15:17.026 DEBUG:teuthology.task.install:project ceph 2023-01-25T04:15:17.027 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}} 2023-01-25T04:15:17.027 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'} 2023-01-25T04:15:17.027 INFO:teuthology.task.install:Using flavor: default 2023-01-25T04:15:17.034 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-01-25T04:15:17.035 INFO:teuthology.task.install:extra packages: [] 2023-01-25T04:15:17.035 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T04:15:17.101 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T04:15:17.103 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:15:17.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'tag': None, 'wait_for_package': False} 2023-01-25T04:15:17.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T04:15:17.184 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T04:15:17.186 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:15:17.186 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:15:17.334 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/ 2023-01-25T04:15:17.334 INFO:teuthology.task.install.rpm:Package version is 18.0.0-1959.gb0d25925 2023-01-25T04:15:17.470 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-01-23-0926/b0d2592529629ab4f86fed5991e556a87dcda563/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-01-25T04:15:17.470 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:15:17.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-01-25T04:15:17.498 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2023-01-25T04:15:17.499 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b0d2592529629ab4f86fed5991e556a87dcda563/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-01-25T04:15:17.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-01-25T04:15:17.646 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-01-25T04:15:17.712 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2023-01-25T04:15:17.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2023-01-25T04:15:18.105 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:15:18.403 INFO:teuthology.orchestra.run.smithi016.stdout:44 files removed 2023-01-25T04:15:18.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw 2023-01-25T04:15:18.802 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:15:19.619 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 248 kB/s | 80 kB 00:00 2023-01-25T04:15:19.912 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 36 kB/s | 9.3 kB 00:00 2023-01-25T04:15:20.179 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 6.1 kB/s | 1.5 kB 00:00 2023-01-25T04:15:20.355 INFO:teuthology.orchestra.run.smithi016.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-01-25T04:15:21.764 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 9.6 MB/s | 13 MB 00:01 2023-01-25T04:15:25.039 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 401 kB/s | 24 kB 00:00 2023-01-25T04:15:26.016 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 MB/s | 53 MB 00:00 2023-01-25T04:15:34.942 INFO:teuthology.orchestra.run.smithi016.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 55 MB/s | 47 MB 00:00 2023-01-25T04:15:46.852 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:15:46.855 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T04:15:46.855 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:15:46.856 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T04:15:46.856 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:15:46.856 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 13 M 2023-01-25T04:15:46.856 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2023-01-25T04:15:46.856 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:15:46.857 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.6 M 2023-01-25T04:15:46.857 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:15:46.857 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 5.2 M 2023-01-25T04:15:46.857 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 M 2023-01-25T04:15:46.857 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 24 k 2023-01-25T04:15:46.858 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-01-25T04:15:46.858 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 768 k 2023-01-25T04:15:46.858 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-01-25T04:15:46.858 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-01-25T04:15:46.858 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 474 k 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.5 M 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 45 k 2023-01-25T04:15:46.859 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 115 k 2023-01-25T04:15:46.860 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 190 k 2023-01-25T04:15:46.860 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-01-25T04:15:46.860 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 372 k 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 364 k 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 107 k 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:15:46.861 INFO:teuthology.orchestra.run.smithi016.stdout:==================================================================================================== 2023-01-25T04:15:46.862 INFO:teuthology.orchestra.run.smithi016.stdout:Install 21 Packages 2023-01-25T04:15:46.862 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 2 Packages 2023-01-25T04:15:46.862 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:15:46.863 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 62 M 2023-01-25T04:15:46.863 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:15:47.550 INFO:teuthology.orchestra.run.smithi016.stdout:(1/23): ceph-base-18.0.0-1959.gb0d25925.el8.x86 7.5 MB/s | 5.2 MB 00:00 2023-01-25T04:15:47.608 INFO:teuthology.orchestra.run.smithi016.stdout:(2/23): ceph-selinux-18.0.0-1959.gb0d25925.el8. 417 kB/s | 24 kB 00:00 2023-01-25T04:15:47.709 INFO:teuthology.orchestra.run.smithi016.stdout:(3/23): libcephfs2-18.0.0-1959.gb0d25925.el8.x8 7.5 MB/s | 768 kB 00:00 2023-01-25T04:15:47.809 INFO:teuthology.orchestra.run.smithi016.stdout:(4/23): libradosstriper1-18.0.0-1959.gb0d25925. 4.6 MB/s | 474 kB 00:00 2023-01-25T04:15:48.501 INFO:teuthology.orchestra.run.smithi016.stdout:(5/23): ceph-radosgw-18.0.0-1959.gb0d25925.el8. 7.9 MB/s | 13 MB 00:01 2023-01-25T04:15:48.617 INFO:teuthology.orchestra.run.smithi016.stdout:(6/23): librgw2-18.0.0-1959.gb0d25925.el8.x86_6 8.0 MB/s | 6.5 MB 00:00 2023-01-25T04:15:48.646 INFO:teuthology.orchestra.run.smithi016.stdout:(7/23): python3-ceph-argparse-18.0.0-1959.gb0d2 315 kB/s | 45 kB 00:00 2023-01-25T04:15:48.668 INFO:teuthology.orchestra.run.smithi016.stdout:(8/23): python3-ceph-common-18.0.0-1959.gb0d259 2.2 MB/s | 115 kB 00:00 2023-01-25T04:15:48.702 INFO:teuthology.orchestra.run.smithi016.stdout:(9/23): python3-cephfs-18.0.0-1959.gb0d25925.el 3.2 MB/s | 190 kB 00:00 2023-01-25T04:15:48.728 INFO:teuthology.orchestra.run.smithi016.stdout:(10/23): python3-rados-18.0.0-1959.gb0d25925.el 6.2 MB/s | 372 kB 00:00 2023-01-25T04:15:48.761 INFO:teuthology.orchestra.run.smithi016.stdout:(11/23): python3-rbd-18.0.0-1959.gb0d25925.el8. 6.1 MB/s | 364 kB 00:00 2023-01-25T04:15:48.795 INFO:teuthology.orchestra.run.smithi016.stdout:(12/23): python3-rgw-18.0.0-1959.gb0d25925.el8. 1.6 MB/s | 107 kB 00:00 2023-01-25T04:15:48.962 INFO:teuthology.orchestra.run.smithi016.stdout:(13/23): gperftools-libs-2.7-9.el8.x86_64.rpm 1.5 MB/s | 306 kB 00:00 2023-01-25T04:15:49.037 INFO:teuthology.orchestra.run.smithi016.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-01-25T04:15:49.062 INFO:teuthology.orchestra.run.smithi016.stdout:(15/23): liboath-2.6.2-3.el8.x86_64.rpm 220 kB/s | 59 kB 00:00 2023-01-25T04:15:49.330 INFO:teuthology.orchestra.run.smithi016.stdout:(16/23): ceph-common-18.0.0-1959.gb0d25925.el8. 9.9 MB/s | 24 MB 00:02 2023-01-25T04:15:49.354 INFO:teuthology.orchestra.run.smithi016.stdout:(17/23): librabbitmq-0.9.0-3.el8.x86_64.rpm 160 kB/s | 47 kB 00:00 2023-01-25T04:15:49.405 INFO:teuthology.orchestra.run.smithi016.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 1.0 MB/s | 44 kB 00:00 2023-01-25T04:15:49.448 INFO:teuthology.orchestra.run.smithi016.stdout:(19/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 2.1 MB/s | 259 kB 00:00 2023-01-25T04:15:49.489 INFO:teuthology.orchestra.run.smithi016.stdout:(20/23): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2023-01-25T04:15:49.514 INFO:teuthology.orchestra.run.smithi016.stdout:(21/23): librdkafka-0.11.4-3.el8.x86_64.rpm 3.2 MB/s | 354 kB 00:00 2023-01-25T04:15:49.773 INFO:teuthology.orchestra.run.smithi016.stdout:(22/23): librados2-18.0.0-1959.gb0d25925.el8.x8 11 MB/s | 3.6 MB 00:00 2023-01-25T04:15:49.868 INFO:teuthology.orchestra.run.smithi016.stdout:(23/23): librbd1-18.0.0-1959.gb0d25925.el8.x86_ 9.4 MB/s | 3.6 MB 00:00 2023-01-25T04:15:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:15:49.870 INFO:teuthology.orchestra.run.smithi016.stdout:Total 20 MB/s | 62 MB 00:03 2023-01-25T04:15:49.914 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:15:49.962 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:15:49.962 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:15:50.324 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:15:50.328 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:15:51.569 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:15:51.787 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/25 2023-01-25T04:15:52.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:15:52.416 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 2/25 2023-01-25T04:15:52.605 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:15:52.692 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:15:52.945 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:15:53.173 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/25 2023-01-25T04:15:53.371 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:15:53.871 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 6/25 2023-01-25T04:15:54.088 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:15:54.183 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:15:54.338 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:15:54.432 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2023-01-25T04:15:54.586 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/25 2023-01-25T04:15:54.725 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 10/25 2023-01-25T04:15:54.965 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:15:55.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:15:55.363 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:15:55.438 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 13/25 2023-01-25T04:15:55.590 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 14/25 2023-01-25T04:15:55.825 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 15/25 2023-01-25T04:15:56.689 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 16/25 2023-01-25T04:15:56.904 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:15:56.979 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 17/25 2023-01-25T04:15:57.224 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 18/25 2023-01-25T04:15:57.446 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 19/25 2023-01-25T04:16:01.619 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:16:01.879 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:16:02.480 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 20/25 2023-01-25T04:16:02.788 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:16:02.875 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 21/25 2023-01-25T04:16:03.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:16:22.497 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:16:22.497 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2023-01-25T04:16:22.497 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2023-01-25T04:16:22.498 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-01-25T04:16:22.498 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /run 2023-01-25T04:16:22.498 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2023-01-25T04:16:22.498 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2023-01-25T04:16:22.498 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2023-01-25T04:16:22.499 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2023-01-25T04:16:22.499 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /dev 2023-01-25T04:16:22.499 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:24.060 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:16:24.082 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 23/25 2023-01-25T04:16:24.083 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:16:24.312 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2023-01-25T04:16:24.312 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:16:24.795 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:16:24.796 INFO:teuthology.orchestra.run.smithi016.stdout:[/usr/lib/tmpfiles.d/journal-nocow.conf:26] Failed to resolve specifier: uninitialized /etc detected, skipping 2023-01-25T04:16:24.796 INFO:teuthology.orchestra.run.smithi016.stdout:All rules containing unresolvable specifiers will be skipped. 2023-01-25T04:16:24.796 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:24.953 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 1/25 2023-01-25T04:16:24.954 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2/25 2023-01-25T04:16:24.954 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 3/25 2023-01-25T04:16:24.954 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 4/25 2023-01-25T04:16:24.954 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 5/25 2023-01-25T04:16:24.955 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_6 6/25 2023-01-25T04:16:24.955 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 7/25 2023-01-25T04:16:24.955 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8. 8/25 2023-01-25T04:16:24.955 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x8 9/25 2023-01-25T04:16:24.955 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 10/25 2023-01-25T04:16:24.956 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 11/25 2023-01-25T04:16:24.956 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 12/25 2023-01-25T04:16:24.956 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 13/25 2023-01-25T04:16:24.956 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2023-01-25T04:16:24.956 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2023-01-25T04:16:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2023-01-25T04:16:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2023-01-25T04:16:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 18/25 2023-01-25T04:16:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2023-01-25T04:16:24.957 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 20/25 2023-01-25T04:16:24.958 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 21/25 2023-01-25T04:16:24.958 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 22/25 2023-01-25T04:16:24.958 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2023-01-25T04:16:24.958 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 24/25 2023-01-25T04:16:26.388 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2023-01-25T04:16:26.388 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:52 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:16:29.082 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:16:29.395 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:29.396 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2023-01-25T04:16:29.396 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.396 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.397 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:16:29.397 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.398 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.398 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.398 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.398 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-01-25T04:16:29.399 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.399 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.2-3.el8.x86_64 2023-01-25T04:16:29.399 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-01-25T04:16:29.400 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.400 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-01-25T04:16:29.400 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.401 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.3.1-3.el8.x86_64 2023-01-25T04:16:29.401 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-01-25T04:16:29.401 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-01-25T04:16:29.402 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.403 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:29.403 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.13.0-2.el8.x86_64 2023-01-25T04:16:29.404 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:29.405 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:16:29.586 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-test 2023-01-25T04:16:29.954 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:16:30.803 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:00:56 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:16:32.327 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:16:32.329 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T04:16:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:16:32.329 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T04:16:32.329 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:16:32.329 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 47 M 2023-01-25T04:16:32.330 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:16:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-01-25T04:16:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 137 k 2023-01-25T04:16:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-01-25T04:16:32.330 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-01-25T04:16:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-01-25T04:16:32.331 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:32.331 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:16:32.331 INFO:teuthology.orchestra.run.smithi016.stdout:============================================================================================ 2023-01-25T04:16:32.331 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2023-01-25T04:16:32.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:32.332 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 48 M 2023-01-25T04:16:32.332 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 218 M 2023-01-25T04:16:32.332 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:16:32.364 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-01-25T04:16:32.465 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 3.5 MB/s | 323 kB 00:00 2023-01-25T04:16:32.498 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): oniguruma-6.8.2-2.el8.x86_64.rpm 5.5 MB/s | 187 kB 00:00 2023-01-25T04:16:32.532 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): jq-1.6-3.el8.x86_64.rpm 6.0 MB/s | 202 kB 00:00 2023-01-25T04:16:32.582 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): libcephsqlite-18.0.0-1959.gb0d25925.el8. 546 kB/s | 137 kB 00:00 2023-01-25T04:16:34.707 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): ceph-test-18.0.0-1959.gb0d25925.el8.x86_ 20 MB/s | 47 MB 00:02 2023-01-25T04:16:34.709 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:16:34.709 INFO:teuthology.orchestra.run.smithi016.stdout:Total 20 MB/s | 48 MB 00:02 2023-01-25T04:16:34.748 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:16:34.772 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:16:34.772 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:16:35.043 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:16:35.047 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:16:35.496 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:16:35.633 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:16:35.713 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 1/6 2023-01-25T04:16:35.939 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : jq-1.6-3.el8.x86_64 2/6 2023-01-25T04:16:36.092 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2023-01-25T04:16:36.287 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2023-01-25T04:16:36.474 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:16:41.386 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 5/6 2023-01-25T04:16:41.654 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:16:41.894 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 6/6 2023-01-25T04:16:41.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 1/6 2023-01-25T04:16:41.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2/6 2023-01-25T04:16:41.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2023-01-25T04:16:41.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2023-01-25T04:16:41.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 5/6 2023-01-25T04:16:43.036 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : jq-1.6-3.el8.x86_64 6/6 2023-01-25T04:16:43.036 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:09 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:16:45.701 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:16:45.899 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:45.899 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:16:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: jq-1.6-3.el8.x86_64 2023-01-25T04:16:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:16:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-01-25T04:16:45.900 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-1.el8.x86_64 2023-01-25T04:16:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-01-25T04:16:45.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:45.901 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:16:46.040 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph 2023-01-25T04:16:46.394 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:16:47.307 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:13 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:16:48.882 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:16:48.886 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T04:16:48.886 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:16:48.886 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T04:16:48.886 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:16:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 6.4 k 2023-01-25T04:16:48.887 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:16:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 2.3 M 2023-01-25T04:16:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 1.5 M 2023-01-25T04:16:48.887 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 236 k 2023-01-25T04:16:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.9 M 2023-01-25T04:16:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 17 M 2023-01-25T04:16:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-01-25T04:16:48.888 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-01-25T04:16:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-01-25T04:16:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-01-25T04:16:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-01-25T04:16:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-01-25T04:16:48.889 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-01-25T04:16:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-01-25T04:16:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-01-25T04:16:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-01-25T04:16:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-01-25T04:16:48.890 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-01-25T04:16:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-01-25T04:16:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-01-25T04:16:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-01-25T04:16:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-01-25T04:16:48.891 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-01-25T04:16:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-01-25T04:16:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-01-25T04:16:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-01-25T04:16:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-01-25T04:16:48.892 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2023-01-25T04:16:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-01-25T04:16:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-01-25T04:16:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-01-25T04:16:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-01-25T04:16:48.893 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:48.894 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:16:48.894 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================= 2023-01-25T04:16:48.894 INFO:teuthology.orchestra.run.smithi016.stdout:Install 32 Packages 2023-01-25T04:16:48.894 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:16:48.895 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 28 M 2023-01-25T04:16:48.895 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 110 M 2023-01-25T04:16:48.895 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:16:49.029 INFO:teuthology.orchestra.run.smithi016.stdout:(1/32): ceph-18.0.0-1959.gb0d25925.el8.x86_64.r 47 kB/s | 6.4 kB 00:00 2023-01-25T04:16:49.212 INFO:teuthology.orchestra.run.smithi016.stdout:(2/32): ceph-mgr-18.0.0-1959.gb0d25925.el8.x86_ 4.8 MB/s | 1.5 MB 00:00 2023-01-25T04:16:49.254 INFO:teuthology.orchestra.run.smithi016.stdout:(3/32): ceph-mds-18.0.0-1959.gb0d25925.el8.x86_ 6.3 MB/s | 2.3 MB 00:00 2023-01-25T04:16:49.362 INFO:teuthology.orchestra.run.smithi016.stdout:(4/32): ceph-mgr-modules-core-18.0.0-1959.gb0d2 2.1 MB/s | 236 kB 00:00 2023-01-25T04:16:49.505 INFO:teuthology.orchestra.run.smithi016.stdout:(5/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 307 kB/s | 44 kB 00:00 2023-01-25T04:16:49.613 INFO:teuthology.orchestra.run.smithi016.stdout:(6/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 1.7 MB/s | 185 kB 00:00 2023-01-25T04:16:49.672 INFO:teuthology.orchestra.run.smithi016.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.9 MB/s | 173 kB 00:00 2023-01-25T04:16:49.774 INFO:teuthology.orchestra.run.smithi016.stdout:(8/32): python3-cherrypy-18.4.0-1.el8.noarch.rp 3.7 MB/s | 384 kB 00:00 2023-01-25T04:16:49.839 INFO:teuthology.orchestra.run.smithi016.stdout:(9/32): ceph-mon-18.0.0-1959.gb0d25925.el8.x86_ 4.8 MB/s | 3.9 MB 00:00 2023-01-25T04:16:49.864 INFO:teuthology.orchestra.run.smithi016.stdout:(10/32): python3-jaraco-6.2-6.el8.noarch.rpm 123 kB/s | 11 kB 00:00 2023-01-25T04:16:49.889 INFO:teuthology.orchestra.run.smithi016.stdout:(11/32): python3-jaraco-functools-2.0-4.el8.noa 350 kB/s | 18 kB 00:00 2023-01-25T04:16:49.940 INFO:teuthology.orchestra.run.smithi016.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2023-01-25T04:16:50.115 INFO:teuthology.orchestra.run.smithi016.stdout:(13/32): ceph-osd-18.0.0-1959.gb0d25925.el8.x86 19 MB/s | 17 MB 00:00 2023-01-25T04:16:50.140 INFO:teuthology.orchestra.run.smithi016.stdout:(14/32): python3-logutils-0.3.5-11.el8.noarch.r 176 kB/s | 49 kB 00:00 2023-01-25T04:16:50.187 INFO:teuthology.orchestra.run.smithi016.stdout:(15/32): python3-pecan-1.3.2-9.el8.noarch.rpm 1.1 MB/s | 283 kB 00:00 2023-01-25T04:16:50.207 INFO:teuthology.orchestra.run.smithi016.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 280 kB/s | 19 kB 00:00 2023-01-25T04:16:50.232 INFO:teuthology.orchestra.run.smithi016.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 532 kB/s | 24 kB 00:00 2023-01-25T04:16:50.258 INFO:teuthology.orchestra.run.smithi016.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 580 kB/s | 29 kB 00:00 2023-01-25T04:16:50.283 INFO:teuthology.orchestra.run.smithi016.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 546 kB/s | 27 kB 00:00 2023-01-25T04:16:50.300 INFO:teuthology.orchestra.run.smithi016.stdout:(20/32): python3-portend-2.6-1.el8.noarch.rpm 84 kB/s | 16 kB 00:00 2023-01-25T04:16:50.368 INFO:teuthology.orchestra.run.smithi016.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.9 MB/s | 251 kB 00:00 2023-01-25T04:16:50.401 INFO:teuthology.orchestra.run.smithi016.stdout:(22/32): python3-waitress-1.2.1-2.el8.1.noarch. 1.7 MB/s | 241 kB 00:00 2023-01-25T04:16:50.426 INFO:teuthology.orchestra.run.smithi016.stdout:(23/32): python3-webtest-2.0.33-1.el8.noarch.rp 680 kB/s | 85 kB 00:00 2023-01-25T04:16:50.443 INFO:teuthology.orchestra.run.smithi016.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 304 kB/s | 23 kB 00:00 2023-01-25T04:16:50.502 INFO:teuthology.orchestra.run.smithi016.stdout:(25/32): python3-pycparser-2.14-14.el8.noarch.r 1.4 MB/s | 109 kB 00:00 2023-01-25T04:16:50.527 INFO:teuthology.orchestra.run.smithi016.stdout:(26/32): python3-cffi-1.11.5-5.el8.x86_64.rpm 1.8 MB/s | 238 kB 00:00 2023-01-25T04:16:50.552 INFO:teuthology.orchestra.run.smithi016.stdout:(27/32): python3-pytz-2017.2-9.el8.noarch.rpm 1.0 MB/s | 54 kB 00:00 2023-01-25T04:16:50.607 INFO:teuthology.orchestra.run.smithi016.stdout:(28/32): python3-markupsafe-0.23-19.el8.x86_64. 485 kB/s | 39 kB 00:00 2023-01-25T04:16:50.637 INFO:teuthology.orchestra.run.smithi016.stdout:(29/32): python3-mako-1.0.6-13.el8.noarch.rpm 1.8 MB/s | 157 kB 00:00 2023-01-25T04:16:50.662 INFO:teuthology.orchestra.run.smithi016.stdout:(30/32): python3-cryptography-3.2.1-5.el8.x86_6 2.5 MB/s | 559 kB 00:00 2023-01-25T04:16:50.687 INFO:teuthology.orchestra.run.smithi016.stdout:(31/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 2.0 MB/s | 103 kB 00:00 2023-01-25T04:16:50.720 INFO:teuthology.orchestra.run.smithi016.stdout:(32/32): python3-werkzeug-0.12.2-4.el8.noarch.r 4.0 MB/s | 457 kB 00:00 2023-01-25T04:16:50.722 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:16:50.723 INFO:teuthology.orchestra.run.smithi016.stdout:Total 15 MB/s | 28 MB 00:01 2023-01-25T04:16:50.817 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:16:50.857 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:16:50.857 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:16:51.074 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:16:51.078 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:16:52.201 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:16:52.863 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2023-01-25T04:16:53.351 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2023-01-25T04:16:53.578 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 3/32 2023-01-25T04:16:53.809 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 4/32 2023-01-25T04:16:54.048 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 5/32 2023-01-25T04:16:54.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 6/32 2023-01-25T04:16:54.523 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 7/32 2023-01-25T04:16:54.916 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 8/32 2023-01-25T04:16:55.243 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 9/32 2023-01-25T04:16:55.405 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 10/32 2023-01-25T04:16:55.602 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 11/32 2023-01-25T04:16:55.806 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 12/32 2023-01-25T04:16:56.019 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2023-01-25T04:16:56.187 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 14/32 2023-01-25T04:16:56.335 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2023-01-25T04:16:56.501 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2023-01-25T04:16:56.658 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2023-01-25T04:16:56.828 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2023-01-25T04:16:57.071 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2023-01-25T04:16:57.301 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2023-01-25T04:16:57.478 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2023-01-25T04:16:57.766 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2023-01-25T04:16:58.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2023-01-25T04:16:58.467 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2023-01-25T04:16:58.824 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2023-01-25T04:16:59.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2023-01-25T04:16:59.449 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 27/32 2023-01-25T04:16:59.597 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:17:01.189 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 28/32 2023-01-25T04:17:01.389 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:17:01.797 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 29/32 2023-01-25T04:17:01.980 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:17:02.205 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 30/32 2023-01-25T04:17:02.404 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:17:02.423 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 31/32 2023-01-25T04:17:02.527 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:17:02.892 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 32/32 2023-01-25T04:17:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 1/32 2023-01-25T04:17:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2/32 2023-01-25T04:17:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 3/32 2023-01-25T04:17:02.893 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 4/32 2023-01-25T04:17:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 5/32 2023-01-25T04:17:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8. 6/32 2023-01-25T04:17:02.894 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2023-01-25T04:17:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2023-01-25T04:17:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2023-01-25T04:17:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2023-01-25T04:17:02.895 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2023-01-25T04:17:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2023-01-25T04:17:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2023-01-25T04:17:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2023-01-25T04:17:02.896 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2023-01-25T04:17:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2023-01-25T04:17:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2023-01-25T04:17:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2023-01-25T04:17:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2023-01-25T04:17:02.897 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 21/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 25/32 2023-01-25T04:17:02.898 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 26/32 2023-01-25T04:17:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 27/32 2023-01-25T04:17:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 28/32 2023-01-25T04:17:02.899 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 29/32 2023-01-25T04:17:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 30/32 2023-01-25T04:17:02.900 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 31/32 2023-01-25T04:17:04.443 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 32/32 2023-01-25T04:17:04.443 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:30 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:07.032 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:17:07.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:07.235 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:17:07.235 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:07.235 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:07.235 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:07.235 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:07.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:07.236 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:07.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-01-25T04:17:07.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-01-25T04:17:07.236 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-01-25T04:17:07.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-01-25T04:17:07.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-01-25T04:17:07.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-01-25T04:17:07.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-6.2-6.el8.noarch 2023-01-25T04:17:07.237 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-01-25T04:17:07.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-01-25T04:17:07.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.0.6-13.el8.noarch 2023-01-25T04:17:07.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-01-25T04:17:07.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-01-25T04:17:07.238 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-01-25T04:17:07.239 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-2.6-1.el8.noarch 2023-01-25T04:17:07.239 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-01-25T04:17:07.239 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.14-14.el8.noarch 2023-01-25T04:17:07.239 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2017.2-9.el8.noarch 2023-01-25T04:17:07.239 INFO:teuthology.orchestra.run.smithi016.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-01-25T04:17:07.240 INFO:teuthology.orchestra.run.smithi016.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-01-25T04:17:07.241 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-01-25T04:17:07.241 INFO:teuthology.orchestra.run.smithi016.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-01-25T04:17:07.241 INFO:teuthology.orchestra.run.smithi016.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2023-01-25T04:17:07.241 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-01-25T04:17:07.242 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-01-25T04:17:07.242 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-01-25T04:17:07.242 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-01-25T04:17:07.242 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:07.242 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:07.395 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-base 2023-01-25T04:17:07.761 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:08.652 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:34 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:10.087 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-base-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:17:10.206 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:10.207 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:17:10.207 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:10.348 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install cephadm 2023-01-25T04:17:10.713 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:11.565 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:37 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:13.129 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:13.130 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:13.130 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:17:13.131 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:13.131 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:17:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 82 k 2023-01-25T04:17:13.131 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:13.132 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:17:13.132 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:13.132 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:17:13.132 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:13.132 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 82 k 2023-01-25T04:17:13.133 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 374 k 2023-01-25T04:17:13.133 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:17:13.336 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm-18.0.0-1959.gb0d25925.el8.noarch.rpm 402 kB/s | 82 kB 00:00 2023-01-25T04:17:13.337 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:17:13.337 INFO:teuthology.orchestra.run.smithi016.stdout:Total 400 kB/s | 82 kB 00:00 2023-01-25T04:17:13.337 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:17:13.346 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:17:13.346 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:17:13.348 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:17:13.352 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:17:13.399 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:17:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:17:15.707 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:17:15.883 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:17:17.017 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:17:17.017 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:43 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:19.658 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:17:19.852 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:19.853 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:17:19.853 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:19.853 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:19.853 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:20.005 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-immutable-object-cache 2023-01-25T04:17:20.383 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:21.239 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:47 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:22.819 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:22.820 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:22.820 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T04:17:22.821 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:22.821 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:17:22.821 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 166 k 2023-01-25T04:17:22.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:22.821 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 166 k 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 503 k 2023-01-25T04:17:22.822 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:17:23.033 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-immutable-object-cache-18.0.0-1959.gb0d259 782 kB/s | 166 kB 00:00 2023-01-25T04:17:23.034 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:17:23.034 INFO:teuthology.orchestra.run.smithi016.stdout:Total 777 kB/s | 166 kB 00:00 2023-01-25T04:17:23.034 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:17:23.048 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:17:23.049 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:17:23.102 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:17:23.106 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:17:23.236 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:17:23.393 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:17:23.765 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:17:24.830 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925. 1/1 2023-01-25T04:17:24.830 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:50 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:27.501 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:17:27.696 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:27.696 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:17:27.696 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:17:27.697 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:27.697 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:27.829 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr 2023-01-25T04:17:28.196 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:29.084 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:55 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:30.521 INFO:teuthology.orchestra.run.smithi016.stdout:Package ceph-mgr-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:17:30.640 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:17:30.641 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:30.772 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-dashboard 2023-01-25T04:17:31.139 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:31.954 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:01:57 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:33.501 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:33.502 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T04:17:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:17:33.503 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T04:17:33.503 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:17:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 1.7 M 2023-01-25T04:17:33.503 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:17:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 23 k 2023-01-25T04:17:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 15 k 2023-01-25T04:17:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:17:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-01-25T04:17:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-01-25T04:17:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:33.505 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:17:33.505 INFO:teuthology.orchestra.run.smithi016.stdout:=================================================================================================== 2023-01-25T04:17:33.505 INFO:teuthology.orchestra.run.smithi016.stdout:Install 6 Packages 2023-01-25T04:17:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:33.506 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 2.0 M 2023-01-25T04:17:33.506 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 8.2 M 2023-01-25T04:17:33.506 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:17:33.664 INFO:teuthology.orchestra.run.smithi016.stdout:(1/6): ceph-grafana-dashboards-18.0.0-1959.gb0d 144 kB/s | 23 kB 00:00 2023-01-25T04:17:33.831 INFO:teuthology.orchestra.run.smithi016.stdout:(2/6): ceph-prometheus-alerts-18.0.0-1959.gb0d2 45 kB/s | 15 kB 00:00 2023-01-25T04:17:33.940 INFO:teuthology.orchestra.run.smithi016.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 122 kB/s | 34 kB 00:00 2023-01-25T04:17:33.982 INFO:teuthology.orchestra.run.smithi016.stdout:(4/6): ceph-mgr-dashboard-18.0.0-1959.gb0d25925 3.6 MB/s | 1.7 MB 00:00 2023-01-25T04:17:34.016 INFO:teuthology.orchestra.run.smithi016.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.0 MB/s | 196 kB 00:00 2023-01-25T04:17:34.041 INFO:teuthology.orchestra.run.smithi016.stdout:(6/6): python3-jwt-1.6.1-2.el8.noarch.rpm 423 kB/s | 43 kB 00:00 2023-01-25T04:17:34.043 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:17:34.043 INFO:teuthology.orchestra.run.smithi016.stdout:Total 3.8 MB/s | 2.0 MB 00:00 2023-01-25T04:17:34.055 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:17:34.065 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:17:34.065 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:17:34.126 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:17:34.130 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:17:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:17:34.560 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2023-01-25T04:17:34.775 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2023-01-25T04:17:34.944 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2023-01-25T04:17:35.123 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 4/6 2023-01-25T04:17:35.473 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 5/6 2023-01-25T04:17:35.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:17:35.963 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 6/6 2023-01-25T04:17:35.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8. 1/6 2023-01-25T04:17:35.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarc 2/6 2023-01-25T04:17:35.964 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.n 3/6 2023-01-25T04:17:35.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2023-01-25T04:17:35.965 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2023-01-25T04:17:37.032 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2023-01-25T04:17:37.032 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:03 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:39.729 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:17:39.926 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:39.927 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:17:39.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:39.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:39.927 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:39.928 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-01-25T04:17:39.928 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-01-25T04:17:39.928 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.4.1-12.el8.noarch 2023-01-25T04:17:39.928 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:39.929 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:40.099 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-diskprediction-local 2023-01-25T04:17:40.466 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:41.286 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:07 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:42.817 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:42.818 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T04:17:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:17:42.819 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T04:17:42.819 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:17:42.819 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 7.4 M 2023-01-25T04:17:42.819 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:17:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-01-25T04:17:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-01-25T04:17:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-01-25T04:17:42.820 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-01-25T04:17:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-01-25T04:17:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-01-25T04:17:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-01-25T04:17:42.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:42.821 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:17:42.822 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================================= 2023-01-25T04:17:42.822 INFO:teuthology.orchestra.run.smithi016.stdout:Install 8 Packages 2023-01-25T04:17:42.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 35 M 2023-01-25T04:17:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 210 M 2023-01-25T04:17:42.823 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:17:42.897 INFO:teuthology.orchestra.run.smithi016.stdout:(1/8): libgfortran-8.5.0-10.1.el8_6.x86_64.rpm 8.3 MB/s | 644 kB 00:00 2023-01-25T04:17:42.922 INFO:teuthology.orchestra.run.smithi016.stdout:(2/8): libquadmath-8.5.0-10.1.el8_6.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2023-01-25T04:17:43.048 INFO:teuthology.orchestra.run.smithi016.stdout:(3/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 30 MB/s | 3.7 MB 00:00 2023-01-25T04:17:43.090 INFO:teuthology.orchestra.run.smithi016.stdout:(4/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 5.2 MB/s | 225 kB 00:00 2023-01-25T04:17:43.276 INFO:teuthology.orchestra.run.smithi016.stdout:(5/8): openblas-threads-0.3.15-3.el8.x86_64.rpm 26 MB/s | 4.9 MB 00:00 2023-01-25T04:17:43.449 INFO:teuthology.orchestra.run.smithi016.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.5.0+10 25 MB/s | 14 MB 00:00 2023-01-25T04:17:43.541 INFO:teuthology.orchestra.run.smithi016.stdout:(7/8): ceph-mgr-diskprediction-local-18.0.0-195 10 MB/s | 7.4 MB 00:00 2023-01-25T04:17:43.633 INFO:teuthology.orchestra.run.smithi016.stdout:(8/8): openblas-0.3.15-3.el8.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2023-01-25T04:17:43.633 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:17:43.634 INFO:teuthology.orchestra.run.smithi016.stdout:Total 44 MB/s | 35 MB 00:00 2023-01-25T04:17:43.879 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:17:43.899 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:17:43.899 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:17:44.208 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:17:44.212 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:17:44.778 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:17:44.956 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:17:45.098 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 1/8 2023-01-25T04:17:45.240 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:17:45.896 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:17:46.056 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:17:46.728 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 3/8 2023-01-25T04:17:46.881 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:17:47.609 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 4/8 2023-01-25T04:17:48.238 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:17:48.440 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:17:50.336 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:17:51.613 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 7/8 2023-01-25T04:17:51.772 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:17:52.051 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 8/8 2023-01-25T04:17:52.051 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d2592 1/8 2023-01-25T04:17:52.051 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 2/8 2023-01-25T04:17:52.051 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 3/8 2023-01-25T04:17:52.052 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d 4/8 2023-01-25T04:17:52.052 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2023-01-25T04:17:52.052 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2023-01-25T04:17:52.052 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 7/8 2023-01-25T04:17:53.158 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 8/8 2023-01-25T04:17:53.158 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:19 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:55.633 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:55.834 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-01-25T04:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.15-3.el8.x86_64 2023-01-25T04:17:55.835 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-01-25T04:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-01-25T04:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-01-25T04:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-01-25T04:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:55.836 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:17:55.975 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-rook 2023-01-25T04:17:56.344 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:17:57.157 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:23 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:17:58.714 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:17:58.716 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T04:17:58.716 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:17:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T04:17:58.717 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:17:58.717 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 47 k 2023-01-25T04:17:58.718 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:17:58.718 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-01-25T04:17:58.718 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-01-25T04:17:58.718 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-01-25T04:17:58.718 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-01-25T04:17:58.719 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-01-25T04:17:58.719 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-01-25T04:17:58.719 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-01-25T04:17:58.719 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-01-25T04:17:58.720 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-01-25T04:17:58.720 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-01-25T04:17:58.720 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-01-25T04:17:58.720 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-01-25T04:17:58.721 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:17:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================================== 2023-01-25T04:17:58.721 INFO:teuthology.orchestra.run.smithi016.stdout:Install 13 Packages 2023-01-25T04:17:58.722 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:17:58.722 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 1.9 M 2023-01-25T04:17:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 25 M 2023-01-25T04:17:58.723 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:17:58.955 INFO:teuthology.orchestra.run.smithi016.stdout:(1/13): ceph-mgr-rook-18.0.0-1959.gb0d25925.el8 201 kB/s | 47 kB 00:00 2023-01-25T04:18:01.955 INFO:teuthology.orchestra.run.smithi016.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 5.0 kB/s | 16 kB 00:03 2023-01-25T04:18:01.980 INFO:teuthology.orchestra.run.smithi016.stdout:(3/13): python3-cachetools-3.1.1-4.el8.noarch.r 10 kB/s | 33 kB 00:03 2023-01-25T04:18:01.998 INFO:teuthology.orchestra.run.smithi016.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 27 kB/s | 81 kB 00:03 2023-01-25T04:18:02.048 INFO:teuthology.orchestra.run.smithi016.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 1.2 MB/s | 61 kB 00:00 2023-01-25T04:18:02.073 INFO:teuthology.orchestra.run.smithi016.stdout:(6/13): python3-rsa-4.8-1.el8.noarch.rpm 667 kB/s | 61 kB 00:00 2023-01-25T04:18:02.173 INFO:teuthology.orchestra.run.smithi016.stdout:(7/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 1.2 MB/s | 155 kB 00:00 2023-01-25T04:18:02.199 INFO:teuthology.orchestra.run.smithi016.stdout:(8/13): python3-requests-oauthlib-1.0.0-1.el8.n 341 kB/s | 43 kB 00:00 2023-01-25T04:18:02.232 INFO:teuthology.orchestra.run.smithi016.stdout:(9/13): python3-kubernetes-11.0.0-6.el8.noarch. 4.2 MB/s | 1.2 MB 00:00 2023-01-25T04:18:02.249 INFO:teuthology.orchestra.run.smithi016.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 1.4 MB/s | 110 kB 00:00 2023-01-25T04:18:02.266 INFO:teuthology.orchestra.run.smithi016.stdout:(11/13): python3-jsonpatch-1.21-2.el8.noarch.rp 396 kB/s | 27 kB 00:00 2023-01-25T04:18:02.308 INFO:teuthology.orchestra.run.smithi016.stdout:(12/13): python3-jsonpointer-1.10-11.el8.noarch 605 kB/s | 20 kB 00:00 2023-01-25T04:18:02.333 INFO:teuthology.orchestra.run.smithi016.stdout:(13/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2023-01-25T04:18:02.334 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:02.334 INFO:teuthology.orchestra.run.smithi016.stdout:Total 546 kB/s | 1.9 MB 00:03 2023-01-25T04:18:02.403 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:02.420 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:02.421 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:02.498 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:02.502 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:02.995 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:03.207 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2023-01-25T04:18:03.434 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.8-1.el8.noarch 2/13 2023-01-25T04:18:03.887 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2023-01-25T04:18:04.020 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 4/13 2023-01-25T04:18:04.243 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 5/13 2023-01-25T04:18:04.487 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2023-01-25T04:18:04.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2023-01-25T04:18:04.996 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2023-01-25T04:18:05.212 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2023-01-25T04:18:05.406 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2023-01-25T04:18:06.584 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2023-01-25T04:18:06.839 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2023-01-25T04:18:06.970 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:18:07.271 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 13/13 2023-01-25T04:18:07.271 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 1/13 2023-01-25T04:18:07.272 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2023-01-25T04:18:07.272 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2023-01-25T04:18:07.273 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2023-01-25T04:18:07.273 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2023-01-25T04:18:07.273 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 6/13 2023-01-25T04:18:07.273 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2023-01-25T04:18:07.273 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2023-01-25T04:18:07.274 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2023-01-25T04:18:07.274 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 10/13 2023-01-25T04:18:07.274 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 11/13 2023-01-25T04:18:07.274 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 12/13 2023-01-25T04:18:08.385 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 13/13 2023-01-25T04:18:08.386 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:34 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:11.168 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:11.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:11.369 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:11.369 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:18:11.369 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-01-25T04:18:11.369 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-01-25T04:18:11.370 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-01-25T04:18:11.370 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-01-25T04:18:11.370 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-01-25T04:18:11.370 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-01-25T04:18:11.370 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-01-25T04:18:11.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-01-25T04:18:11.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-01-25T04:18:11.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-01-25T04:18:11.371 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.8-1.el8.noarch 2023-01-25T04:18:11.372 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-01-25T04:18:11.372 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:11.372 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:11.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-mgr-cephadm 2023-01-25T04:18:11.882 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:12.690 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:38 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:14.281 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:14.282 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T04:18:14.282 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:18:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T04:18:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:18:14.283 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 120 k 2023-01-25T04:18:14.283 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:18:14.283 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-01-25T04:18:14.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-01-25T04:18:14.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-01-25T04:18:14.284 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-01-25T04:18:14.284 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:14.284 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:18:14.285 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================================== 2023-01-25T04:18:14.285 INFO:teuthology.orchestra.run.smithi016.stdout:Install 5 Packages 2023-01-25T04:18:14.285 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:14.286 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 6.0 M 2023-01-25T04:18:14.286 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 26 M 2023-01-25T04:18:14.286 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:18:14.424 INFO:teuthology.orchestra.run.smithi016.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 432 kB/s | 60 kB 00:00 2023-01-25T04:18:14.500 INFO:teuthology.orchestra.run.smithi016.stdout:(2/5): ceph-mgr-cephadm-18.0.0-1959.gb0d25925.e 559 kB/s | 120 kB 00:00 2023-01-25T04:18:14.541 INFO:teuthology.orchestra.run.smithi016.stdout:(3/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 4.5 MB/s | 538 kB 00:00 2023-01-25T04:18:14.575 INFO:teuthology.orchestra.run.smithi016.stdout:(4/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.6 MB/s | 476 kB 00:00 2023-01-25T04:18:14.667 INFO:teuthology.orchestra.run.smithi016.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 29 MB/s | 4.8 MB 00:00 2023-01-25T04:18:14.668 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:14.668 INFO:teuthology.orchestra.run.smithi016.stdout:Total 15 MB/s | 6.0 MB 00:00 2023-01-25T04:18:14.722 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:14.735 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:14.736 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:14.818 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:14.822 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:15.873 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:16.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2023-01-25T04:18:16.414 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2023-01-25T04:18:16.759 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:18:17.075 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2023-01-25T04:18:17.261 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:18:17.489 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 5/5 2023-01-25T04:18:17.489 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 1/5 2023-01-25T04:18:17.490 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2023-01-25T04:18:17.490 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2023-01-25T04:18:17.490 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 4/5 2023-01-25T04:18:18.565 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 5/5 2023-01-25T04:18:18.565 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:44 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:21.068 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:21.268 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:21.268 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:18:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-01-25T04:18:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.5.1-7.el8.noarch 2023-01-25T04:18:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-01-25T04:18:21.269 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-01-25T04:18:21.270 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:21.270 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:21.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-fuse 2023-01-25T04:18:21.756 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:22.587 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:48 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:24.138 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:24.139 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:24.139 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:18:24.140 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:24.140 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:18:24.140 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse 2023-01-25T04:18:24.141 INFO:teuthology.orchestra.run.smithi016.stdout: x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 845 k 2023-01-25T04:18:24.141 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2023-01-25T04:18:24.141 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-01-25T04:18:24.141 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:24.142 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:18:24.142 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:24.142 INFO:teuthology.orchestra.run.smithi016.stdout:Install 2 Packages 2023-01-25T04:18:24.142 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:24.142 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 927 k 2023-01-25T04:18:24.143 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 2.9 M 2023-01-25T04:18:24.143 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:18:24.204 INFO:teuthology.orchestra.run.smithi016.stdout:(1/2): fuse-2.9.7-15.el8.x86_64.rpm 1.3 MB/s | 83 kB 00:00 2023-01-25T04:18:24.438 INFO:teuthology.orchestra.run.smithi016.stdout:(2/2): ceph-fuse-18.0.0-1959.gb0d25925.el8.x86_ 2.8 MB/s | 845 kB 00:00 2023-01-25T04:18:24.438 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:24.439 INFO:teuthology.orchestra.run.smithi016.stdout:Total 3.0 MB/s | 927 kB 00:00 2023-01-25T04:18:24.450 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:24.473 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:24.474 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:24.552 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:24.556 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:24.734 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:25.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.7-15.el8.x86_64 1/2 2023-01-25T04:18:25.159 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:18:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2/2 2023-01-25T04:18:25.493 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/2 2023-01-25T04:18:26.985 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 2/2 2023-01-25T04:18:26.985 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:52 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:29.964 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:30.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:30.161 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:30.161 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 fuse-2.9.7-15.el8.x86_64 2023-01-25T04:18:30.161 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:30.162 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:30.336 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-volume 2023-01-25T04:18:30.716 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:31.552 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:02:57 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:33.109 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:33.110 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2023-01-25T04:18:33.110 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:33.110 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:18:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:18.0.0-1959.gb0d25925.el8 ceph-noarch 263 k 2023-01-25T04:18:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:33.111 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:18:33.111 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:33.111 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:18:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 263 k 2023-01-25T04:18:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 1.3 M 2023-01-25T04:18:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:18:33.675 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-volume-18.0.0-1959.gb0d25925.el8.noarch.rp 466 kB/s | 263 kB 00:00 2023-01-25T04:18:33.675 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:33.676 INFO:teuthology.orchestra.run.smithi016.stdout:Total 465 kB/s | 263 kB 00:00 2023-01-25T04:18:33.676 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:33.687 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:33.688 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:33.707 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:33.711 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:18:34.520 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:18:35.614 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 1/1 2023-01-25T04:18:35.614 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:01 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:38.136 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:38.335 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:38.336 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:38.336 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:18.0.0-1959.gb0d25925.el8.noarch 2023-01-25T04:18:38.336 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:38.336 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:38.471 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados-devel 2023-01-25T04:18:38.852 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:39.729 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:05 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:41.280 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:41.281 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:41.281 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T04:18:41.281 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:41.281 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 139 k 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:18:41.282 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:41.283 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 139 k 2023-01-25T04:18:41.283 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 512 k 2023-01-25T04:18:41.283 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:18:41.482 INFO:teuthology.orchestra.run.smithi016.stdout:librados-devel-18.0.0-1959.gb0d25925.el8.x86_64 695 kB/s | 139 kB 00:00 2023-01-25T04:18:41.483 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:41.483 INFO:teuthology.orchestra.run.smithi016.stdout:Total 691 kB/s | 139 kB 00:00 2023-01-25T04:18:41.483 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:41.489 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:41.489 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:41.547 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:41.551 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:41.673 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:41.939 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:42.197 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:43.269 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:43.269 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:09 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:46.014 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:46.215 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:46.215 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:46.215 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:18:46.216 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:46.216 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:46.369 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs2 2023-01-25T04:18:46.748 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:47.578 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:13 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:49.025 INFO:teuthology.orchestra.run.smithi016.stdout:Package libcephfs2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:18:49.147 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:18:49.148 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:49.268 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install libcephfs-devel 2023-01-25T04:18:49.645 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:50.479 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:16 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:51.958 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:51.959 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T04:18:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:51.959 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:18:51.959 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 30 k 2023-01-25T04:18:51.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:18:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:18:51.960 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:18:51.961 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 30 k 2023-01-25T04:18:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 137 k 2023-01-25T04:18:51.961 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:18:52.147 INFO:teuthology.orchestra.run.smithi016.stdout:libcephfs-devel-18.0.0-1959.gb0d25925.el8.x86_6 163 kB/s | 30 kB 00:00 2023-01-25T04:18:52.148 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:18:52.148 INFO:teuthology.orchestra.run.smithi016.stdout:Total 162 kB/s | 30 kB 00:00 2023-01-25T04:18:52.148 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:18:52.152 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:18:52.152 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:18:52.163 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:18:52.167 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:18:52.222 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:18:52.437 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:52.642 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:53.665 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:18:53.666 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:19 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:56.168 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:18:56.367 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:56.368 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:18:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:18:56.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:18:56.368 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:56.505 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librados2 2023-01-25T04:18:56.880 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:18:57.750 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:23 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:18:59.179 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:18:59.296 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:18:59.297 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:18:59.298 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:18:59.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install librbd1 2023-01-25T04:18:59.807 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:00.619 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:26 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:02.047 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:19:02.166 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:02.167 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:19:02.167 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:02.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rados 2023-01-25T04:19:02.656 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:03.473 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:29 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:04.903 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rados-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:19:05.021 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:05.022 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:19:05.023 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:05.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rgw 2023-01-25T04:19:05.539 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:06.376 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:32 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:07.821 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rgw-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:19:07.940 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:07.941 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:19:07.941 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:08.061 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-cephfs 2023-01-25T04:19:08.438 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:09.262 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:35 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:10.698 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-cephfs-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:19:10.818 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:10.819 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:19:10.820 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:10.940 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install python3-rbd 2023-01-25T04:19:11.315 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:12.141 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:38 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:13.525 INFO:teuthology.orchestra.run.smithi016.stdout:Package python3-rbd-2:18.0.0-1959.gb0d25925.el8.x86_64 is already installed. 2023-01-25T04:19:13.643 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:Nothing to do. 2023-01-25T04:19:13.645 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:13.765 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-fuse 2023-01-25T04:19:14.140 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:14.958 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:40 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:16.512 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:16.513 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:16.514 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-01-25T04:19:16.514 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:16.514 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:19:16.514 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 94 k 2023-01-25T04:19:16.514 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:16.515 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:19:16.515 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:16.515 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:19:16.516 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:16.516 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 94 k 2023-01-25T04:19:16.516 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 280 k 2023-01-25T04:19:16.516 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:19:16.724 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-fuse-18.0.0-1959.gb0d25925.el8.x86_64.rpm 448 kB/s | 94 kB 00:00 2023-01-25T04:19:16.725 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:19:16.725 INFO:teuthology.orchestra.run.smithi016.stdout:Total 446 kB/s | 94 kB 00:00 2023-01-25T04:19:16.726 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:19:16.737 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:19:16.737 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:19:16.794 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:19:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:19:16.937 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:19:17.084 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:17.313 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:18.393 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:18.394 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:44 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:21.225 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:19:21.425 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:21.425 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:19:21.426 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:19:21.426 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:21.426 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:21.594 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-mirror 2023-01-25T04:19:21.971 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:22.783 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:48 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:24.305 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:24.306 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:24.306 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repo Size 2023-01-25T04:19:24.306 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:24.306 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:19:24.306 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 3.4 M 2023-01-25T04:19:24.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:24.307 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:19:24.307 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:24.307 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:19:24.307 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:24.308 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 3.4 M 2023-01-25T04:19:24.308 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 16 M 2023-01-25T04:19:24.308 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:19:24.678 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-mirror-18.0.0-1959.gb0d25925.el8.x86_64.rpm 9.3 MB/s | 3.4 MB 00:00 2023-01-25T04:19:24.678 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:19:24.679 INFO:teuthology.orchestra.run.smithi016.stdout:Total 9.3 MB/s | 3.4 MB 00:00 2023-01-25T04:19:24.679 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:19:24.692 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:19:24.693 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:19:24.763 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:19:24.767 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:19:25.230 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:19:25.402 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:25.778 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:26.978 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:26.978 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:52 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:29.704 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:19:29.900 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:29.900 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:19:29.901 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:19:29.901 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:29.901 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:30.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install rbd-nbd 2023-01-25T04:19:30.431 INFO:teuthology.orchestra.run.smithi016.stdout:Updating Subscription Management repositories. 2023-01-25T04:19:31.263 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:03:57 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:32.831 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2023-01-25T04:19:32.832 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:32.832 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2023-01-25T04:19:32.832 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:32.833 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2023-01-25T04:19:32.833 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:18.0.0-1959.gb0d25925.el8 ceph 179 k 2023-01-25T04:19:32.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:32.833 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2023-01-25T04:19:32.833 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2023-01-25T04:19:32.834 INFO:teuthology.orchestra.run.smithi016.stdout:Install 1 Package 2023-01-25T04:19:32.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:32.834 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 179 k 2023-01-25T04:19:32.834 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 568 k 2023-01-25T04:19:32.834 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2023-01-25T04:19:33.040 INFO:teuthology.orchestra.run.smithi016.stdout:rbd-nbd-18.0.0-1959.gb0d25925.el8.x86_64.rpm 867 kB/s | 179 kB 00:00 2023-01-25T04:19:33.041 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2023-01-25T04:19:33.041 INFO:teuthology.orchestra.run.smithi016.stdout:Total 862 kB/s | 179 kB 00:00 2023-01-25T04:19:33.041 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2023-01-25T04:19:33.053 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2023-01-25T04:19:33.054 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2023-01-25T04:19:33.112 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2023-01-25T04:19:33.116 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2023-01-25T04:19:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2023-01-25T04:19:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:34.712 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 1/1 2023-01-25T04:19:34.712 INFO:teuthology.orchestra.run.smithi016.stdout:Last metadata expiration check: 0:04:00 ago on Wed 25 Jan 2023 04:15:34 AM UTC. 2023-01-25T04:19:37.220 INFO:teuthology.orchestra.run.smithi016.stdout:Installed products updated. 2023-01-25T04:19:37.419 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:37.419 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2023-01-25T04:19:37.420 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:18.0.0-1959.gb0d25925.el8.x86_64 2023-01-25T04:19:37.420 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:37.420 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2023-01-25T04:19:37.555 DEBUG:teuthology.parallel:result is None 2023-01-25T04:19:37.556 DEBUG:teuthology.orchestra.run.smithi016:> sudo lsb_release -is 2023-01-25T04:19:37.600 INFO:teuthology.orchestra.run.smithi016.stdout:RedHatEnterprise 2023-01-25T04:19:37.601 DEBUG:teuthology.misc:System to be installed: RedHatEnterprise 2023-01-25T04:19:37.602 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:19:37.857 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-01-25T04:19:37.880 INFO:teuthology.orchestra.run.smithi016.stdout:18.0.0-1959.gb0d25925.el8 2023-01-25T04:19:37.880 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-1959.gb0d25925.el8 2023-01-25T04:19:37.881 INFO:teuthology.task.install:The correct ceph version 18.0.0-1959.gb0d25925 is installed. 2023-01-25T04:19:37.881 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-01-25T04:19:37.882 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:37.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-01-25T04:19:37.948 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-01-25T04:19:37.948 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:37.949 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2023-01-25T04:19:38.015 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-01-25T04:19:38.082 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-01-25T04:19:38.082 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:38.083 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2023-01-25T04:19:38.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-01-25T04:19:38.216 INFO:teuthology.run_tasks:Running task ceph... 2023-01-25T04:19:38.278 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-01-25T04:19:38.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /var/log/ceph 2023-01-25T04:19:38.306 INFO:tasks.ceph:Disabling ceph logrotate... 2023-01-25T04:19:38.307 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/logrotate.d/ceph 2023-01-25T04:19:38.373 INFO:tasks.ceph:Creating extra log directories... 2023-01-25T04:19:38.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-01-25T04:19:38.449 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-01-25T04:19:38.449 INFO:tasks.ceph:config {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-01-25T04:19:38.450 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi/7136725', 'branch': 'wip-yuri-testing-2023-01-23-0926', 'description': 'rados/singleton-nomsgr/{all/health-warnings mon_election/classic rados supported-random-distro$/{rhel_8}}', 'email': 'yweinstein@radhat.com', 'first_in_suite': False, 'job_id': '7136725', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2023-01-24_22:20:59-rados-wip-yuri-testing-2023-01-23-0926-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-01-23-0926'}, 'ceph': {'conf': {'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'workunit': {'branch': 'wip-yuri-testing-2023-01-23-0926', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 71, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'osd.8', 'osd.9', 'client.0']], 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'sleep_before_teardown': 0, 'suite': 'rados', 'suite_branch': 'wip-yuri-testing-2023-01-23-0926', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_b0d2592529629ab4f86fed5991e556a87dcda563/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'targets': {'smithi016.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'global': {'mon election default strategy': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'log-ignorelist': ['but it is still running', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'pre-mgr-commands': ['sudo ceph config set mgr mgr_pool false --force'], 'flavor': 'default', 'sha1': 'b0d2592529629ab4f86fed5991e556a87dcda563', 'cluster': 'ceph'}}, {'workunit': {'clients': {'all': ['rados/test_health_warnings.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8e1bba12e584b7ae912e92e94afe9c075ab51fa3', 'timestamp': '2023-01-24_22:20:59', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.17735'} 2023-01-25T04:19:38.450 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-01-25T04:19:38.497 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m0777 -- /var/run/ceph 2023-01-25T04:19:38.564 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:38.565 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2023-01-25T04:19:38.622 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-01-25T04:19:38.622 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2023-01-25T04:19:38.679 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-01-25T04:19:38.679 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T04:19:38.679 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 350790 Links: 1 2023-01-25T04:19:38.680 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T04:19:38.680 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T04:19:38.680 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 04:19:25.574208146 +0000 2023-01-25T04:19:38.680 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 04:14:48.285684655 +0000 2023-01-25T04:19:38.681 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 04:14:48.285684655 +0000 2023-01-25T04:19:38.681 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T04:19:38.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-01-25T04:19:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T04:19:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T04:19:38.745 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000101172 s, 5.1 MB/s 2023-01-25T04:19:38.746 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-01-25T04:19:38.804 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2023-01-25T04:19:38.862 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-01-25T04:19:38.862 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T04:19:38.862 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 350838 Links: 1 2023-01-25T04:19:38.863 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T04:19:38.863 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T04:19:38.863 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 04:19:25.575208130 +0000 2023-01-25T04:19:38.863 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 04:14:48.797676592 +0000 2023-01-25T04:19:38.863 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 04:14:48.797676592 +0000 2023-01-25T04:19:38.864 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T04:19:38.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-01-25T04:19:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T04:19:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T04:19:38.928 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000103582 s, 4.9 MB/s 2023-01-25T04:19:38.929 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-01-25T04:19:38.987 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2023-01-25T04:19:39.045 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-01-25T04:19:39.045 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T04:19:39.046 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 350970 Links: 1 2023-01-25T04:19:39.046 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T04:19:39.046 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T04:19:39.046 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 04:19:25.575208130 +0000 2023-01-25T04:19:39.046 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 04:14:49.301668655 +0000 2023-01-25T04:19:39.047 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 04:14:49.301668655 +0000 2023-01-25T04:19:39.047 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T04:19:39.047 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-01-25T04:19:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T04:19:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T04:19:39.114 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000109928 s, 4.7 MB/s 2023-01-25T04:19:39.115 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-01-25T04:19:39.173 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2023-01-25T04:19:39.230 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-01-25T04:19:39.230 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-01-25T04:19:39.231 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 6h/6d Inode: 351062 Links: 1 2023-01-25T04:19:39.231 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-01-25T04:19:39.231 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2023-01-25T04:19:39.231 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2023-01-25 04:19:25.575208130 +0000 2023-01-25T04:19:39.231 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2023-01-25 04:14:49.813660592 +0000 2023-01-25T04:19:39.232 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2023-01-25 04:14:49.813660592 +0000 2023-01-25T04:19:39.232 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: - 2023-01-25T04:19:39.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-01-25T04:19:39.295 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2023-01-25T04:19:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2023-01-25T04:19:39.296 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000122034 s, 4.2 MB/s 2023-01-25T04:19:39.297 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-01-25T04:19:39.355 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:39.356 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-01-25T04:19:39.356 INFO:tasks.ceph:Generating config... 2023-01-25T04:19:39.357 INFO:tasks.ceph:[osd] osd max object name len = 400 2023-01-25T04:19:39.357 INFO:tasks.ceph:[osd] osd max object namespace len = 64 2023-01-25T04:19:39.358 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2023-01-25T04:19:39.358 INFO:tasks.ceph:[osd] debug ms = 1 2023-01-25T04:19:39.358 INFO:tasks.ceph:[osd] debug osd = 20 2023-01-25T04:19:39.359 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2023-01-25T04:19:39.359 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2023-01-25T04:19:39.359 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2023-01-25T04:19:39.359 INFO:tasks.ceph:[osd] osd op queue = debug_random 2023-01-25T04:19:39.360 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2023-01-25T04:19:39.360 INFO:tasks.ceph:[global] mon election default strategy = 1 2023-01-25T04:19:39.360 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-01-25T04:19:39.361 INFO:tasks.ceph:[mgr] debug ms = 1 2023-01-25T04:19:39.361 INFO:tasks.ceph:[mon] debug mon = 20 2023-01-25T04:19:39.361 INFO:tasks.ceph:[mon] debug ms = 1 2023-01-25T04:19:39.361 INFO:tasks.ceph:[mon] debug paxos = 20 2023-01-25T04:19:39.362 INFO:tasks.ceph:[mon] mon scrub interval = 300 2023-01-25T04:19:39.362 INFO:tasks.ceph:Setting up mon.a... 2023-01-25T04:19:39.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-01-25T04:19:39.447 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.keyring 2023-01-25T04:19:39.450 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-01-25T04:19:39.495 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T04:19:39.522 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.16')] 2023-01-25T04:19:39.523 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.16', 'mon election default strategy': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'osd max object name len': 400, 'osd max object namespace len': 64, 'bluestore zero block detection': True, 'debug ms': 1, 'debug osd': 20, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd mclock override recovery settings': True, 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2023-01-25T04:19:39.524 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:39.524 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T04:19:39.579 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.16 --print /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:39.661 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:39.662 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: generated fsid 94576b13-ca6e-4c0d-acc4-16dd0344f762 2023-01-25T04:19:39.662 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = pacific 2023-01-25T04:19:39.662 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 0 2023-01-25T04:19:39.663 INFO:teuthology.orchestra.run.smithi016.stdout:fsid 94576b13-ca6e-4c0d-acc4-16dd0344f762 2023-01-25T04:19:39.663 INFO:teuthology.orchestra.run.smithi016.stdout:last_changed 2023-01-25T04:19:39.661527+0000 2023-01-25T04:19:39.663 INFO:teuthology.orchestra.run.smithi016.stdout:created 2023-01-25T04:19:39.661527+0000 2023-01-25T04:19:39.663 INFO:teuthology.orchestra.run.smithi016.stdout:min_mon_release 16 (pacific) 2023-01-25T04:19:39.663 INFO:teuthology.orchestra.run.smithi016.stdout:election_strategy: 1 2023-01-25T04:19:39.664 INFO:teuthology.orchestra.run.smithi016.stdout:0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2023-01-25T04:19:39.664 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2023-01-25T04:19:39.665 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-01-25T04:19:39.722 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 94576b13-ca6e-4c0d-acc4-16dd0344f762... 2023-01-25T04:19:39.723 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-01-25T04:19:39.809 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2023-01-25T04:19:39.809 INFO:teuthology.orchestra.run.smithi016.stdout: chdir = "" 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: auth supported = cephx 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: filestore xattr use omap = true 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.810 INFO:teuthology.orchestra.run.smithi016.stdout: mon clock drift allowed = 1.000 2023-01-25T04:19:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: osd crush chooseleaf type = 0 2023-01-25T04:19:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: auth debug = true 2023-01-25T04:19:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.811 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on old message = true 2023-01-25T04:19:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: ms die on bug = true 2023-01-25T04:19:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon max pg per osd = 10000 # >= luminous 2023-01-25T04:19:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: mon pg warn max object skew = 0 2023-01-25T04:19:39.812 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: # disable pg_autoscaler by default for new pools 2023-01-25T04:19:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd_pool_default_pg_autoscale_mode = off 2023-01-25T04:19:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default size = 2 2023-01-25T04:19:39.813 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow primary affinity = true 2023-01-25T04:19:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd allow pg remap = true 2023-01-25T04:19:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on legacy crush tunables = false 2023-01-25T04:19:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on crush straw calc version zero = false 2023-01-25T04:19:39.814 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on no sortbitwise = false 2023-01-25T04:19:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on osd down out interval zero = false 2023-01-25T04:19:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon warn on too few osds = false 2023-01-25T04:19:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-01-25T04:19:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_pool_no_redundancy = false 2023-01-25T04:19:39.815 INFO:teuthology.orchestra.run.smithi016.stdout: mon_allow_pool_size_one = true 2023-01-25T04:19:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-01-25T04:19:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: osd default data pool replay window = 5 2023-01-25T04:19:39.816 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon allow pool delete = true 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon cluster log file level = debug 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: debug asserts on shutdown = true 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon health detail to clog = false 2023-01-25T04:19:39.817 INFO:teuthology.orchestra.run.smithi016.stdout: mon host = 172.21.15.16 2023-01-25T04:19:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: mon election default strategy = 1 2023-01-25T04:19:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = 94576b13-ca6e-4c0d-acc4-16dd0344f762 2023-01-25T04:19:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.818 INFO:teuthology.orchestra.run.smithi016.stdout:[osd] 2023-01-25T04:19:39.818 INFO:teuthology.orchestra.run.smithi016.stdout: osd journal size = 100 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub load threshold = 5.0 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd scrub max interval = 600 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd recover clone overlap = true 2023-01-25T04:19:39.819 INFO:teuthology.orchestra.run.smithi016.stdout: osd recovery max chunk = 1048576 2023-01-25T04:19:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug shutdown = true 2023-01-25T04:19:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug op order = true 2023-01-25T04:19:39.820 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify stray on activate = true 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: osd open classes on start = true 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug pg log writeout = true 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: osd deep scrub update digest min age = 30 2023-01-25T04:19:39.821 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.822 INFO:teuthology.orchestra.run.smithi016.stdout: osd map max advance = 10 2023-01-25T04:19:39.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.822 INFO:teuthology.orchestra.run.smithi016.stdout: journal zero on create = true 2023-01-25T04:19:39.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.822 INFO:teuthology.orchestra.run.smithi016.stdout: filestore ondisk finisher threads = 3 2023-01-25T04:19:39.823 INFO:teuthology.orchestra.run.smithi016.stdout: filestore apply finisher threads = 3 2023-01-25T04:19:39.823 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.823 INFO:teuthology.orchestra.run.smithi016.stdout: bdev debug aio = true 2023-01-25T04:19:39.824 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug misdirected ops = true 2023-01-25T04:19:39.825 INFO:teuthology.orchestra.run.smithi016.stdout: osd max object name len = 400 2023-01-25T04:19:39.825 INFO:teuthology.orchestra.run.smithi016.stdout: osd max object namespace len = 64 2023-01-25T04:19:39.825 INFO:teuthology.orchestra.run.smithi016.stdout: bluestore zero block detection = True 2023-01-25T04:19:39.825 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T04:19:39.825 INFO:teuthology.orchestra.run.smithi016.stdout: debug osd = 20 2023-01-25T04:19:39.826 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify cached snaps = True 2023-01-25T04:19:39.826 INFO:teuthology.orchestra.run.smithi016.stdout: osd debug verify missing on start = True 2023-01-25T04:19:39.826 INFO:teuthology.orchestra.run.smithi016.stdout: osd mclock override recovery settings = True 2023-01-25T04:19:39.826 INFO:teuthology.orchestra.run.smithi016.stdout: osd op queue = debug_random 2023-01-25T04:19:39.826 INFO:teuthology.orchestra.run.smithi016.stdout: osd op queue cut off = debug_random 2023-01-25T04:19:39.827 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.827 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr] 2023-01-25T04:19:39.827 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T04:19:39.827 INFO:teuthology.orchestra.run.smithi016.stdout: debug mgr = 20 2023-01-25T04:19:39.827 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-01-25T04:19:39.828 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-01-25T04:19:39.828 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-01-25T04:19:39.828 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-01-25T04:19:39.828 INFO:teuthology.orchestra.run.smithi016.stdout: mgr/telemetry/nag = false 2023-01-25T04:19:39.828 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.829 INFO:teuthology.orchestra.run.smithi016.stdout:[mon] 2023-01-25T04:19:39.829 INFO:teuthology.orchestra.run.smithi016.stdout: debug ms = 1 2023-01-25T04:19:39.829 INFO:teuthology.orchestra.run.smithi016.stdout: debug mon = 20 2023-01-25T04:19:39.829 INFO:teuthology.orchestra.run.smithi016.stdout: debug paxos = 20 2023-01-25T04:19:39.829 INFO:teuthology.orchestra.run.smithi016.stdout: debug auth = 20 2023-01-25T04:19:39.830 INFO:teuthology.orchestra.run.smithi016.stdout: mon data avail warn = 5 2023-01-25T04:19:39.830 INFO:teuthology.orchestra.run.smithi016.stdout: mon mgr mkfs grace = 240 2023-01-25T04:19:39.830 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min pgs per osd = 4 2023-01-25T04:19:39.830 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd reporter subtree level = osd 2023-01-25T04:19:39.830 INFO:teuthology.orchestra.run.smithi016.stdout: mon osd prime pg temp = true 2023-01-25T04:19:39.831 INFO:teuthology.orchestra.run.smithi016.stdout: mon reweight min bytes per osd = 10 2023-01-25T04:19:39.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.831 INFO:teuthology.orchestra.run.smithi016.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-01-25T04:19:39.831 INFO:teuthology.orchestra.run.smithi016.stdout: auth mon ticket ttl = 660 # 11m 2023-01-25T04:19:39.831 INFO:teuthology.orchestra.run.smithi016.stdout: auth service ticket ttl = 240 # 4m 2023-01-25T04:19:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: # don't complain about insecure global_id in the test suite 2023-01-25T04:19:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-01-25T04:19:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-01-25T04:19:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.833 INFO:teuthology.orchestra.run.smithi016.stdout: # 1m isn't quite enough 2023-01-25T04:19:39.833 INFO:teuthology.orchestra.run.smithi016.stdout: mon_down_mkfs_grace = 2m 2023-01-25T04:19:39.833 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.833 INFO:teuthology.orchestra.run.smithi016.stdout: mon_warn_on_filestore_osds = false 2023-01-25T04:19:39.833 INFO:teuthology.orchestra.run.smithi016.stdout: mon scrub interval = 300 2023-01-25T04:19:39.834 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:19:39.834 INFO:teuthology.orchestra.run.smithi016.stdout:[client] 2023-01-25T04:19:39.834 INFO:teuthology.orchestra.run.smithi016.stdout: rgw cache enabled = true 2023-01-25T04:19:39.834 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable ops log = true 2023-01-25T04:19:39.834 INFO:teuthology.orchestra.run.smithi016.stdout: rgw enable usage log = true 2023-01-25T04:19:39.835 INFO:teuthology.orchestra.run.smithi016.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-01-25T04:19:39.835 INFO:teuthology.orchestra.run.smithi016.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-01-25T04:19:39.835 INFO:teuthology.orchestra.run.smithi016.stdout:[mon.a] 2023-01-25T04:19:39.836 INFO:tasks.ceph:Creating admin key on mon.a... 2023-01-25T04:19:39.837 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-01-25T04:19:39.936 INFO:tasks.ceph:Copying monmap to all nodes... 2023-01-25T04:19:39.937 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:39.937 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-01-25T04:19:39.954 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:39.955 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-01-25T04:19:40.011 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:19:40.012 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:40.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.keyring 2023-01-25T04:19:40.012 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-01-25T04:19:40.088 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:19:40.089 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:40.145 INFO:tasks.ceph:Setting up mon nodes... 2023-01-25T04:19:40.146 INFO:tasks.ceph:Setting up mgr nodes... 2023-01-25T04:19:40.146 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T04:19:40.240 INFO:teuthology.orchestra.run.smithi016.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-01-25T04:19:40.242 INFO:tasks.ceph:Setting up mds nodes... 2023-01-25T04:19:40.243 INFO:tasks.ceph_client:Setting up client nodes... 2023-01-25T04:19:40.243 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-01-25T04:19:40.287 INFO:teuthology.orchestra.run.smithi016.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-01-25T04:19:40.300 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-01-25T04:19:40.301 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi016.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}} 2023-01-25T04:19:40.301 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-01-25T04:19:40.329 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:40.329 INFO:tasks.ceph:role: osd.0 2023-01-25T04:19:40.329 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:19:40.330 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-01-25T04:19:40.396 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T04:19:40.396 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T04:19:40.396 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T04:19:40.397 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T04:19:40.397 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T04:19:40.397 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T04:19:40.397 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T04:19:40.397 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T04:19:40.398 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T04:19:40.398 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T04:19:40.419 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T04:19:40.420 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T04:19:40.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-01-25T04:19:40.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-01-25T04:19:40.575 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-01-25T04:19:40.644 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:40.644 INFO:tasks.ceph:role: osd.1 2023-01-25T04:19:40.645 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:19:40.645 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-01-25T04:19:40.710 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T04:19:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T04:19:40.711 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T04:19:40.712 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T04:19:40.712 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T04:19:40.712 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T04:19:40.713 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T04:19:40.713 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T04:19:40.713 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T04:19:40.714 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T04:19:40.733 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T04:19:40.735 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T04:19:40.735 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-01-25T04:19:40.815 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-01-25T04:19:40.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-01-25T04:19:40.954 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:40.954 INFO:tasks.ceph:role: osd.2 2023-01-25T04:19:40.955 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:19:40.955 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-01-25T04:19:41.021 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T04:19:41.021 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T04:19:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T04:19:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T04:19:41.022 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T04:19:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T04:19:41.022 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T04:19:41.023 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T04:19:41.023 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T04:19:41.023 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T04:19:41.043 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T04:19:41.045 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T04:19:41.045 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-01-25T04:19:41.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-01-25T04:19:41.197 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-01-25T04:19:41.265 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.265 INFO:tasks.ceph:role: osd.3 2023-01-25T04:19:41.266 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com 2023-01-25T04:19:41.266 DEBUG:teuthology.orchestra.run.smithi016:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-01-25T04:19:41.332 INFO:teuthology.orchestra.run.smithi016.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-01-25T04:19:41.333 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 attr=2, projid32bit=1 2023-01-25T04:19:41.333 INFO:teuthology.orchestra.run.smithi016.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-01-25T04:19:41.333 INFO:teuthology.orchestra.run.smithi016.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-01-25T04:19:41.333 INFO:teuthology.orchestra.run.smithi016.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-01-25T04:19:41.333 INFO:teuthology.orchestra.run.smithi016.stdout: = sunit=0 swidth=0 blks 2023-01-25T04:19:41.334 INFO:teuthology.orchestra.run.smithi016.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-01-25T04:19:41.334 INFO:teuthology.orchestra.run.smithi016.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-01-25T04:19:41.334 INFO:teuthology.orchestra.run.smithi016.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-01-25T04:19:41.334 INFO:teuthology.orchestra.run.smithi016.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-01-25T04:19:41.355 INFO:teuthology.orchestra.run.smithi016.stdout:Discarding blocks...Done. 2023-01-25T04:19:41.357 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi016.front.sepia.ceph.com -o noatime 2023-01-25T04:19:41.357 DEBUG:teuthology.orchestra.run.smithi016:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-01-25T04:19:41.437 DEBUG:teuthology.orchestra.run.smithi016:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-01-25T04:19:41.508 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-01-25T04:19:41.575 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.576 INFO:tasks.ceph:role: osd.4 2023-01-25T04:19:41.576 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-01-25T04:19:41.642 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.643 INFO:tasks.ceph:role: osd.5 2023-01-25T04:19:41.643 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-01-25T04:19:41.709 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.710 INFO:tasks.ceph:role: osd.6 2023-01-25T04:19:41.710 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-01-25T04:19:41.776 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.776 INFO:tasks.ceph:role: osd.7 2023-01-25T04:19:41.777 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2023-01-25T04:19:41.844 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.844 INFO:tasks.ceph:role: osd.8 2023-01-25T04:19:41.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2023-01-25T04:19:41.910 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-01-25T04:19:41.911 INFO:tasks.ceph:role: osd.9 2023-01-25T04:19:41.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-01-25T04:19:42.025 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-01-25T04:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T04:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T04:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-01-25T04:19:42.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.023+0000 7f665b2f05c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-01-25T04:19:42.549 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:42.547+0000 7f665b2f05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:43.313 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:43.311+0000 7f665b2f05c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:44.106 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-01-25T04:19:44.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-01-25T04:19:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-01-25T04:19:44.245 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T04:19:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T04:19:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-01-25T04:19:44.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.243+0000 7f2d3d0fb5c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-01-25T04:19:44.775 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:44.774+0000 7f2d3d0fb5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:45.539 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:45.537+0000 7f2d3d0fb5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:46.331 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-01-25T04:19:46.363 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-01-25T04:19:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-01-25T04:19:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T04:19:46.457 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T04:19:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-01-25T04:19:46.458 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.455+0000 7fe7535005c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-01-25T04:19:46.984 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:46.983+0000 7fe7535005c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:47.746 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:47.744+0000 7fe7535005c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:48.546 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-01-25T04:19:48.579 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:48.672 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.670+0000 7f4aee88b5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-01-25T04:19:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.670+0000 7f4aee88b5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-01-25T04:19:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.670+0000 7f4aee88b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T04:19:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.670+0000 7f4aee88b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T04:19:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.670+0000 7f4aee88b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-01-25T04:19:48.673 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:48.671+0000 7f4aee88b5c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-01-25T04:19:49.198 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:49.196+0000 7f4aee88b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:49.962 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:49.960+0000 7f4aee88b5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:50.754 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-01-25T04:19:50.787 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:50.880 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.878+0000 7f92005ca5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-01-25T04:19:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.879+0000 7f92005ca5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-01-25T04:19:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.879+0000 7f92005ca5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T04:19:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.879+0000 7f92005ca5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T04:19:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.879+0000 7f92005ca5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-01-25T04:19:50.881 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:50.879+0000 7f92005ca5c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-01-25T04:19:51.409 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:51.408+0000 7f92005ca5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:52.170 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:52.169+0000 7f92005ca5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:52.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-01-25T04:19:52.996 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.089+0000 7f785015e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-01-25T04:19:53.091 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.089+0000 7f785015e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-01-25T04:19:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.089+0000 7f785015e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T04:19:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.089+0000 7f785015e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T04:19:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.089+0000 7f785015e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-01-25T04:19:53.092 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.090+0000 7f785015e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-01-25T04:19:53.621 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:53.619+0000 7f785015e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:54.384 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:54.382+0000 7f785015e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:55.177 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-01-25T04:19:55.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.303+0000 7f751005e5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-01-25T04:19:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.303+0000 7f751005e5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-01-25T04:19:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.303+0000 7f751005e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T04:19:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.303+0000 7f751005e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T04:19:55.306 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.303+0000 7f751005e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-01-25T04:19:55.307 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.304+0000 7f751005e5c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-01-25T04:19:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:55.830+0000 7f751005e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:56.594 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:56.592+0000 7f751005e5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:57.386 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-01-25T04:19:57.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:57.513 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.511+0000 7f63a1f845c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-01-25T04:19:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.511+0000 7f63a1f845c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-01-25T04:19:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.511+0000 7f63a1f845c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T04:19:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.511+0000 7f63a1f845c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T04:19:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.511+0000 7f63a1f845c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-01-25T04:19:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:57.512+0000 7f63a1f845c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-01-25T04:19:58.040 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:58.038+0000 7f63a1f845c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:58.804 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:58.802+0000 7f63a1f845c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:19:59.600 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-01-25T04:19:59.633 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:19:59.726 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.724+0000 7f42f44ce5c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2023-01-25T04:19:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.725+0000 7f42f44ce5c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2023-01-25T04:19:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.725+0000 7f42f44ce5c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-25T04:19:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.725+0000 7f42f44ce5c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-25T04:19:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.725+0000 7f42f44ce5c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2023-01-25T04:19:59.727 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:19:59.725+0000 7f42f44ce5c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2023-01-25T04:20:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:00.258+0000 7f42f44ce5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:20:01.026 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.025+0000 7f42f44ce5c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:20:01.824 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2023-01-25T04:20:01.857 DEBUG:teuthology.orchestra.run.smithi016:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:20:01.952 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.950+0000 7f07465535c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2023-01-25T04:20:01.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.950+0000 7f07465535c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2023-01-25T04:20:01.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.950+0000 7f07465535c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-25T04:20:01.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.950+0000 7f07465535c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-25T04:20:01.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.950+0000 7f07465535c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2023-01-25T04:20:01.953 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:01.951+0000 7f07465535c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2023-01-25T04:20:02.483 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:02.481+0000 7f07465535c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:20:03.246 INFO:teuthology.orchestra.run.smithi016.stderr:2023-01-25T04:20:03.244+0000 7f07465535c0 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-01-25T04:20:04.038 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2023-01-25T04:20:04.071 INFO:tasks.ceph:Reading keys from all nodes... 2023-01-25T04:20:04.071 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.071 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-01-25T04:20:04.142 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.143 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-01-25T04:20:04.213 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-01-25T04:20:04.283 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.283 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-01-25T04:20:04.352 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.352 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-01-25T04:20:04.421 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-01-25T04:20:04.490 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-01-25T04:20:04.559 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.559 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-01-25T04:20:04.626 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.626 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-01-25T04:20:04.694 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.694 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2023-01-25T04:20:04.761 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2023-01-25T04:20:04.828 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:04.829 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-01-25T04:20:04.886 INFO:tasks.ceph:Adding keys to all mons... 2023-01-25T04:20:04.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo tee -a /etc/ceph/ceph.keyring 2023-01-25T04:20:04.952 INFO:teuthology.orchestra.run.smithi016.stdout:[mgr.x] 2023-01-25T04:20:04.952 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDcrdBjW9BNDhAAa1BGyrm77iYD1CG0FkSp7Q== 2023-01-25T04:20:04.952 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.0] 2023-01-25T04:20:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDerdBjU8J4ARAAAj7qgA4+fmrYLKd3zngWyQ== 2023-01-25T04:20:04.953 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.1] 2023-01-25T04:20:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDgrdBjbm2TDhAAIjqVxulLtiFWwoUOGl4hcw== 2023-01-25T04:20:04.953 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.2] 2023-01-25T04:20:04.953 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDirdBjc6YzGxAAbgC2N4PyM2Ezi2hLT8eT8g== 2023-01-25T04:20:04.954 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.3] 2023-01-25T04:20:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDkrdBj+9sMKBAAbbG2hguhCjdKnYqbLEBFiA== 2023-01-25T04:20:04.954 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.4] 2023-01-25T04:20:04.954 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDmrdBjTAR3NBAAhRUfPGy6eq6rpXhUzFS/Vw== 2023-01-25T04:20:04.954 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.5] 2023-01-25T04:20:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDprdBjX3RtBRAAuK7ugxYQO9m9jO2s6Qy20w== 2023-01-25T04:20:04.955 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.6] 2023-01-25T04:20:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDrrdBj0HEtEhAABStbow8um1CecEBRqBb17Q== 2023-01-25T04:20:04.955 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.7] 2023-01-25T04:20:04.955 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDtrdBjAWeVHhAAoc0RqQDh0Pf25/cJ7bfQKA== 2023-01-25T04:20:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.8] 2023-01-25T04:20:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDvrdBjyCtJKxAAcbgCJLowDZeguzlCxNv6qg== 2023-01-25T04:20:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:[osd.9] 2023-01-25T04:20:04.956 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDxrdBjyQ28OBAA/W9xR/5Xmw+SBpmbrnsLDw== 2023-01-25T04:20:04.956 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2023-01-25T04:20:04.957 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQDcrdBjXAcfERAA5fD1IfoBstLym9lsba5mKA== 2023-01-25T04:20:04.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-01-25T04:20:05.051 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.104 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.158 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.212 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.266 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.320 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.375 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.428 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.483 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-01-25T04:20:05.592 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-01-25T04:20:05.650 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-01-25T04:20:05.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-01-25T04:20:05.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-01-25T04:20:05.775 DEBUG:teuthology.orchestra.run.smithi016:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-01-25T04:20:05.802 DEBUG:teuthology.orchestra.run.smithi016:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-01-25T04:20:05.859 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-01-25T04:20:05.859 INFO:tasks.ceph.mon.a:Restarting daemon 2023-01-25T04:20:05.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-01-25T04:20:05.901 INFO:tasks.ceph.mon.a:Started 2023-01-25T04:20:05.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph config set mgr mgr_pool false --force 2023-01-25T04:20:06.121 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-01-25T04:20:06.121 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-01-25T04:20:06.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-01-25T04:20:06.124 INFO:tasks.ceph.mgr.x:Started 2023-01-25T04:20:06.125 INFO:tasks.ceph:Setting crush tunables to default 2023-01-25T04:20:06.125 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd crush tunables default 2023-01-25T04:20:06.320 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:06.319+0000 7f32441bd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-01-25T04:20:06.339 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2023-01-25T04:20:06.350 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-01-25T04:20:06.350 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.351 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-01-25T04:20:06.379 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.380 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-01-25T04:20:06.392 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:06.390+0000 7f32441bd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-01-25T04:20:06.419 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.419 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-01-25T04:20:06.470 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:06.469+0000 7f32441bd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-01-25T04:20:06.485 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.486 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-01-25T04:20:06.554 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.554 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-01-25T04:20:06.555 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:06.553+0000 7f32441bd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-01-25T04:20:06.581 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-01-25T04:20:06.649 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.650 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-01-25T04:20:06.702 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:06.701+0000 7f32441bd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-01-25T04:20:06.716 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.717 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-01-25T04:20:06.784 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.785 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2023-01-25T04:20:06.852 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:20:06.852 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2023-01-25T04:20:06.919 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 843bf900-0cea-4573-b26b-c2fc7fbb0dfc 0 2023-01-25T04:20:07.060 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:07.059+0000 7f32441bd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-01-25T04:20:07.140 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:07.139+0000 7f32441bd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-01-25T04:20:07.178 INFO:teuthology.orchestra.run.smithi016.stdout:0 2023-01-25T04:20:07.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 20016226-4430-4bf5-891a-4ed2415659af 1 2023-01-25T04:20:07.269 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:07.268+0000 7f32441bd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-01-25T04:20:07.366 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:07.364+0000 7f32441bd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-01-25T04:20:07.412 INFO:teuthology.orchestra.run.smithi016.stdout:1 2023-01-25T04:20:07.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 1c73c582-5f9a-4f7b-9631-a6dafe254a79 2 2023-01-25T04:20:07.644 INFO:teuthology.orchestra.run.smithi016.stdout:2 2023-01-25T04:20:07.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 4c0614b6-a46d-4428-8c8c-e87eec7fa624 3 2023-01-25T04:20:07.829 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:07.827+0000 7f32441bd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-01-25T04:20:07.875 INFO:teuthology.orchestra.run.smithi016.stdout:3 2023-01-25T04:20:07.889 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 64fba726-cab2-42b8-81d8-c57bc425dd9c 4 2023-01-25T04:20:08.109 INFO:teuthology.orchestra.run.smithi016.stdout:4 2023-01-25T04:20:08.121 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 01cbab51-a662-4e35-ae3a-9902a3109369 5 2023-01-25T04:20:08.405 INFO:teuthology.orchestra.run.smithi016.stdout:5 2023-01-25T04:20:08.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new b0337d52-03c4-4788-b488-3773e40fa420 6 2023-01-25T04:20:08.454 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:08.453+0000 7f32441bd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-01-25T04:20:08.616 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:08.614+0000 7f32441bd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-01-25T04:20:08.643 INFO:teuthology.orchestra.run.smithi016.stdout:6 2023-01-25T04:20:08.655 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 35b039dc-2c63-46c0-b7cc-2ccca15a0866 7 2023-01-25T04:20:08.924 INFO:teuthology.orchestra.run.smithi016.stdout:7 2023-01-25T04:20:08.937 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new 8d0279b3-fbec-4a30-aaeb-a64c4ad211f1 8 2023-01-25T04:20:09.154 INFO:teuthology.orchestra.run.smithi016.stdout:8 2023-01-25T04:20:09.167 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd new e9a272ed-ec7e-4307-9130-5249fd3fbc16 9 2023-01-25T04:20:09.344 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:09.343+0000 7f32441bd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-01-25T04:20:09.389 INFO:teuthology.orchestra.run.smithi016.stdout:9 2023-01-25T04:20:09.402 INFO:tasks.ceph.osd.0:Restarting daemon 2023-01-25T04:20:09.403 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-01-25T04:20:09.405 INFO:tasks.ceph.osd.0:Started 2023-01-25T04:20:09.405 INFO:tasks.ceph.osd.1:Restarting daemon 2023-01-25T04:20:09.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-01-25T04:20:09.408 INFO:tasks.ceph.osd.1:Started 2023-01-25T04:20:09.408 INFO:tasks.ceph.osd.2:Restarting daemon 2023-01-25T04:20:09.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-01-25T04:20:09.411 INFO:tasks.ceph.osd.2:Started 2023-01-25T04:20:09.411 INFO:tasks.ceph.osd.3:Restarting daemon 2023-01-25T04:20:09.411 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-01-25T04:20:09.413 INFO:tasks.ceph.osd.3:Started 2023-01-25T04:20:09.414 INFO:tasks.ceph.osd.4:Restarting daemon 2023-01-25T04:20:09.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-01-25T04:20:09.416 INFO:tasks.ceph.osd.4:Started 2023-01-25T04:20:09.417 INFO:tasks.ceph.osd.5:Restarting daemon 2023-01-25T04:20:09.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-01-25T04:20:09.421 INFO:tasks.ceph.osd.5:Started 2023-01-25T04:20:09.421 INFO:tasks.ceph.osd.6:Restarting daemon 2023-01-25T04:20:09.421 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-01-25T04:20:09.425 INFO:tasks.ceph.osd.6:Started 2023-01-25T04:20:09.426 INFO:tasks.ceph.osd.7:Restarting daemon 2023-01-25T04:20:09.426 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-01-25T04:20:09.431 INFO:tasks.ceph.osd.7:Started 2023-01-25T04:20:09.432 INFO:tasks.ceph.osd.8:Restarting daemon 2023-01-25T04:20:09.432 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2023-01-25T04:20:09.437 INFO:tasks.ceph.osd.8:Started 2023-01-25T04:20:09.437 INFO:tasks.ceph.osd.9:Restarting daemon 2023-01-25T04:20:09.438 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2023-01-25T04:20:09.439 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:09.434+0000 7f32441bd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-01-25T04:20:09.441 INFO:tasks.ceph.osd.9:Started 2023-01-25T04:20:09.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T04:20:09.667 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:09.666+0000 7f32441bd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-01-25T04:20:09.775 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:09.775 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:20:09.387970+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:20:09.788 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-01-25T04:20:09.788 INFO:tasks.ceph:Waiting for OSDs to come up 2023-01-25T04:20:09.836 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:09.834+0000 7f32441bd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-01-25T04:20:09.860 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:20:09.858+0000 7f411826d5c0 -1 Falling back to public interface 2023-01-25T04:20:09.864 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:20:09.862+0000 7f8c4473b5c0 -1 Falling back to public interface 2023-01-25T04:20:09.882 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:20:09.880+0000 7fba66ddf5c0 -1 Falling back to public interface 2023-01-25T04:20:09.974 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:09.972+0000 7f32441bd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-01-25T04:20:10.122 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:20:10.120+0000 7f8f2510b5c0 -1 Falling back to public interface 2023-01-25T04:20:10.129 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:20:10.127+0000 7f704ca9a5c0 -1 Falling back to public interface 2023-01-25T04:20:10.134 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:20:10.133+0000 7fe5c30a05c0 -1 Falling back to public interface 2023-01-25T04:20:10.135 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:20:10.133+0000 7fa90f5d25c0 -1 Falling back to public interface 2023-01-25T04:20:10.138 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:20:10.136+0000 7f5e6e9af5c0 -1 Falling back to public interface 2023-01-25T04:20:10.164 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:20:10.162+0000 7f8d953485c0 -1 Falling back to public interface 2023-01-25T04:20:10.183 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:20:10.181+0000 7f987744a5c0 -1 Falling back to public interface 2023-01-25T04:20:10.356 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:10.354+0000 7f32441bd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-01-25T04:20:10.516 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:10.514+0000 7f32441bd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-01-25T04:20:10.636 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:10.634+0000 7f32441bd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-01-25T04:20:10.789 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:10.788+0000 7f32441bd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-01-25T04:20:10.793 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T04:20:10.898 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:10.896+0000 7f32441bd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-01-25T04:20:10.988 INFO:teuthology.misc.health.smithi016.stdout: 2023-01-25T04:20:10.988 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":12,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:20:09.387970+0000","last_up_change":"0.000000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:20:11.000 DEBUG:teuthology.misc:0 of 10 OSDs are up 2023-01-25T04:20:11.271 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:11.269+0000 7f32441bd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-01-25T04:20:11.882 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:11.881+0000 7f32441bd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-01-25T04:20:12.616 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:20:12.614+0000 7f8c4473b5c0 -1 osd.0 0 log_to_monitors true 2023-01-25T04:20:12.617 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:20:12.615+0000 7fba66ddf5c0 -1 osd.4 0 log_to_monitors true 2023-01-25T04:20:12.630 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:20:12.629+0000 7f411826d5c0 -1 osd.5 0 log_to_monitors true 2023-01-25T04:20:12.874 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:20:12.872+0000 7fe5c30a05c0 -1 osd.2 0 log_to_monitors true 2023-01-25T04:20:12.874 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:20:12.873+0000 7fa90f5d25c0 -1 osd.7 0 log_to_monitors true 2023-01-25T04:20:12.883 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:20:12.881+0000 7f704ca9a5c0 -1 osd.3 0 log_to_monitors true 2023-01-25T04:20:12.889 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:20:12.887+0000 7f8f2510b5c0 -1 osd.1 0 log_to_monitors true 2023-01-25T04:20:12.896 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:20:12.894+0000 7f5e6e9af5c0 -1 osd.6 0 log_to_monitors true 2023-01-25T04:20:12.901 INFO:tasks.ceph.mgr.x.smithi016.stderr:2023-01-25T04:20:12.900+0000 7f31fc53e700 -1 mgr.server handle_report got status from non-daemon mon.a 2023-01-25T04:20:12.909 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:20:12.908+0000 7f8d953485c0 -1 osd.9 0 log_to_monitors true 2023-01-25T04:20:12.924 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:20:12.922+0000 7f987744a5c0 -1 osd.8 0 log_to_monitors true 2023-01-25T04:20:14.042 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:20:14.040+0000 7f410c625700 -1 osd.5 0 waiting for initial osdmap 2023-01-25T04:20:14.051 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:20:14.050+0000 7f8c38af3700 -1 osd.0 0 waiting for initial osdmap 2023-01-25T04:20:14.052 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:20:14.050+0000 7fe5b7458700 -1 osd.2 0 waiting for initial osdmap 2023-01-25T04:20:14.052 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:20:14.050+0000 7f986b802700 -1 osd.8 0 waiting for initial osdmap 2023-01-25T04:20:14.053 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:20:14.050+0000 7f7040e52700 -1 osd.3 0 waiting for initial osdmap 2023-01-25T04:20:14.053 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:20:14.050+0000 7fba5b197700 -1 osd.4 0 waiting for initial osdmap 2023-01-25T04:20:14.053 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:20:14.050+0000 7f5e62d67700 -1 osd.6 0 waiting for initial osdmap 2023-01-25T04:20:14.054 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:20:14.050+0000 7fa90398a700 -1 osd.7 0 waiting for initial osdmap 2023-01-25T04:20:14.054 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:20:14.053+0000 7f8f194c3700 -1 osd.1 0 waiting for initial osdmap 2023-01-25T04:20:14.055 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:20:14.053+0000 7f4106a04700 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.055 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:20:14.054+0000 7f8d89700700 -1 osd.9 0 waiting for initial osdmap 2023-01-25T04:20:14.056 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:20:14.054+0000 7fe5b2038700 -1 osd.2 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.056 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:20:14.055+0000 7fba55576700 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.058 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:20:14.055+0000 7f8c32ed2700 -1 osd.0 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.058 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:20:14.055+0000 7f703ba32700 -1 osd.3 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.059 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:20:14.055+0000 7f5e5d947700 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.059 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:20:14.055+0000 7f9866be3700 -1 osd.8 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.060 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:20:14.057+0000 7f8f140a3700 -1 osd.1 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.060 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:20:14.058+0000 7fa8fe56a700 -1 osd.7 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:14.060 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:20:14.058+0000 7f8d84ae1700 -1 osd.9 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:20:18.006 DEBUG:teuthology.orchestra.run.smithi016:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-01-25T04:20:18.339 INFO:teuthology.misc.health.smithi016.stdout: 2023-01-25T04:20:18.339 INFO:teuthology.misc.health.smithi016.stdout:{"epoch":16,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:20:16.059337+0000","last_up_change":"2023-01-25T04:20:15.032345+0000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":97132},{"type":"v1","addr":"172.21.15.16:6809","nonce":97132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":97132},{"type":"v1","addr":"172.21.15.16:6811","nonce":97132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":97132},{"type":"v1","addr":"172.21.15.16:6815","nonce":97132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":97132},{"type":"v1","addr":"172.21.15.16:6813","nonce":97132}]},"public_addr":"172.21.15.16:6809/97132","cluster_addr":"172.21.15.16:6811/97132","heartbeat_back_addr":"172.21.15.16:6815/97132","heartbeat_front_addr":"172.21.15.16:6813/97132","state":["exists","up"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":97131},{"type":"v1","addr":"172.21.15.16:6825","nonce":97131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":97131},{"type":"v1","addr":"172.21.15.16:6827","nonce":97131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":97131},{"type":"v1","addr":"172.21.15.16:6831","nonce":97131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":97131},{"type":"v1","addr":"172.21.15.16:6829","nonce":97131}]},"public_addr":"172.21.15.16:6825/97131","cluster_addr":"172.21.15.16:6827/97131","heartbeat_back_addr":"172.21.15.16:6831/97131","heartbeat_front_addr":"172.21.15.16:6829/97131","state":["exists","up"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6841","nonce":97156},{"type":"v1","addr":"172.21.15.16:6843","nonce":97156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6845","nonce":97156},{"type":"v1","addr":"172.21.15.16:6847","nonce":97156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6856","nonce":97156},{"type":"v1","addr":"172.21.15.16:6858","nonce":97156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6850","nonce":97156},{"type":"v1","addr":"172.21.15.16:6853","nonce":97156}]},"public_addr":"172.21.15.16:6843/97156","cluster_addr":"172.21.15.16:6847/97156","heartbeat_back_addr":"172.21.15.16:6858/97156","heartbeat_front_addr":"172.21.15.16:6853/97156","state":["exists","up"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":97139},{"type":"v1","addr":"172.21.15.16:6833","nonce":97139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":97139},{"type":"v1","addr":"172.21.15.16:6835","nonce":97139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":97139},{"type":"v1","addr":"172.21.15.16:6840","nonce":97139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":97139},{"type":"v1","addr":"172.21.15.16:6837","nonce":97139}]},"public_addr":"172.21.15.16:6833/97139","cluster_addr":"172.21.15.16:6835/97139","heartbeat_back_addr":"172.21.15.16:6840/97139","heartbeat_front_addr":"172.21.15.16:6837/97139","state":["exists","up"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":97135},{"type":"v1","addr":"172.21.15.16:6817","nonce":97135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":97135},{"type":"v1","addr":"172.21.15.16:6819","nonce":97135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":97135},{"type":"v1","addr":"172.21.15.16:6823","nonce":97135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":97135},{"type":"v1","addr":"172.21.15.16:6821","nonce":97135}]},"public_addr":"172.21.15.16:6817/97135","cluster_addr":"172.21.15.16:6819/97135","heartbeat_back_addr":"172.21.15.16:6823/97135","heartbeat_front_addr":"172.21.15.16:6821/97135","state":["exists","up"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":97137},{"type":"v1","addr":"172.21.15.16:6801","nonce":97137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":97137},{"type":"v1","addr":"172.21.15.16:6803","nonce":97137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":97137},{"type":"v1","addr":"172.21.15.16:6807","nonce":97137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":97137},{"type":"v1","addr":"172.21.15.16:6805","nonce":97137}]},"public_addr":"172.21.15.16:6801/97137","cluster_addr":"172.21.15.16:6803/97137","heartbeat_back_addr":"172.21.15.16:6807/97137","heartbeat_front_addr":"172.21.15.16:6805/97137","state":["exists","up"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6848","nonce":97157},{"type":"v1","addr":"172.21.15.16:6851","nonce":97157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6854","nonce":97157},{"type":"v1","addr":"172.21.15.16:6857","nonce":97157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6862","nonce":97157},{"type":"v1","addr":"172.21.15.16:6863","nonce":97157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6860","nonce":97157},{"type":"v1","addr":"172.21.15.16:6861","nonce":97157}]},"public_addr":"172.21.15.16:6851/97157","cluster_addr":"172.21.15.16:6857/97157","heartbeat_back_addr":"172.21.15.16:6863/97157","heartbeat_front_addr":"172.21.15.16:6861/97157","state":["exists","up"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6839","nonce":97145},{"type":"v1","addr":"172.21.15.16:6842","nonce":97145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6844","nonce":97145},{"type":"v1","addr":"172.21.15.16:6846","nonce":97145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6855","nonce":97145},{"type":"v1","addr":"172.21.15.16:6859","nonce":97145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6849","nonce":97145},{"type":"v1","addr":"172.21.15.16:6852","nonce":97145}]},"public_addr":"172.21.15.16:6842/97145","cluster_addr":"172.21.15.16:6846/97145","heartbeat_back_addr":"172.21.15.16:6859/97145","heartbeat_front_addr":"172.21.15.16:6852/97145","state":["exists","up"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6872","nonce":97243},{"type":"v1","addr":"172.21.15.16:6873","nonce":97243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6874","nonce":97243},{"type":"v1","addr":"172.21.15.16:6875","nonce":97243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6878","nonce":97243},{"type":"v1","addr":"172.21.15.16:6879","nonce":97243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6876","nonce":97243},{"type":"v1","addr":"172.21.15.16:6877","nonce":97243}]},"public_addr":"172.21.15.16:6873/97243","cluster_addr":"172.21.15.16:6875/97243","heartbeat_back_addr":"172.21.15.16:6879/97243","heartbeat_front_addr":"172.21.15.16:6877/97243","state":["exists","up"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6864","nonce":97237},{"type":"v1","addr":"172.21.15.16:6865","nonce":97237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6866","nonce":97237},{"type":"v1","addr":"172.21.15.16:6867","nonce":97237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6870","nonce":97237},{"type":"v1","addr":"172.21.15.16:6871","nonce":97237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6868","nonce":97237},{"type":"v1","addr":"172.21.15.16:6869","nonce":97237}]},"public_addr":"172.21.15.16:6865/97237","cluster_addr":"172.21.15.16:6867/97237","heartbeat_back_addr":"172.21.15.16:6871/97237","heartbeat_front_addr":"172.21.15.16:6869/97237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.619173+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.900051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.896545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.850003+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.626569+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.630371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.931585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.898826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.893908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.906245+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:20:18.352 DEBUG:teuthology.misc:10 of 10 OSDs are up 2023-01-25T04:20:18.353 INFO:tasks.ceph:Creating RBD pool 2023-01-25T04:20:18.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool create rbd 8 2023-01-25T04:20:19.069 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'rbd' created 2023-01-25T04:20:19.081 DEBUG:teuthology.orchestra.run.smithi016:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-01-25T04:20:20.077 INFO:teuthology.orchestra.run.smithi016.stderr:enabled application 'rbd' on pool 'rbd' 2023-01-25T04:20:20.091 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-01-25T04:20:20.092 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-01-25T04:20:20.092 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-01-25T04:20:20.092 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-01-25T04:20:20.093 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-01-25T04:20:20.484 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:20.498 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6880","nonce":96163},{"type":"v1","addr":"172.21.15.16:6881","nonce":96163}]},"active_addr":"172.21.15.16:6881/96163","active_change":"2023-01-25T04:20:11.895108+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3455145592}]},{"addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":565371929}]},{"addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":296421076}]}]}} 2023-01-25T04:20:20.502 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-01-25T04:20:20.502 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-01-25T04:20:20.502 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T04:20:20.831 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:20.831 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":18,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:20:20.066475+0000","last_up_change":"2023-01-25T04:20:15.032345+0000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T04:20:18.715898+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":97132},{"type":"v1","addr":"172.21.15.16:6809","nonce":97132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":97132},{"type":"v1","addr":"172.21.15.16:6811","nonce":97132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":97132},{"type":"v1","addr":"172.21.15.16:6815","nonce":97132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":97132},{"type":"v1","addr":"172.21.15.16:6813","nonce":97132}]},"public_addr":"172.21.15.16:6809/97132","cluster_addr":"172.21.15.16:6811/97132","heartbeat_back_addr":"172.21.15.16:6815/97132","heartbeat_front_addr":"172.21.15.16:6813/97132","state":["exists","up"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":97131},{"type":"v1","addr":"172.21.15.16:6825","nonce":97131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":97131},{"type":"v1","addr":"172.21.15.16:6827","nonce":97131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":97131},{"type":"v1","addr":"172.21.15.16:6831","nonce":97131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":97131},{"type":"v1","addr":"172.21.15.16:6829","nonce":97131}]},"public_addr":"172.21.15.16:6825/97131","cluster_addr":"172.21.15.16:6827/97131","heartbeat_back_addr":"172.21.15.16:6831/97131","heartbeat_front_addr":"172.21.15.16:6829/97131","state":["exists","up"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6841","nonce":97156},{"type":"v1","addr":"172.21.15.16:6843","nonce":97156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6845","nonce":97156},{"type":"v1","addr":"172.21.15.16:6847","nonce":97156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6856","nonce":97156},{"type":"v1","addr":"172.21.15.16:6858","nonce":97156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6850","nonce":97156},{"type":"v1","addr":"172.21.15.16:6853","nonce":97156}]},"public_addr":"172.21.15.16:6843/97156","cluster_addr":"172.21.15.16:6847/97156","heartbeat_back_addr":"172.21.15.16:6858/97156","heartbeat_front_addr":"172.21.15.16:6853/97156","state":["exists","up"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":97139},{"type":"v1","addr":"172.21.15.16:6833","nonce":97139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":97139},{"type":"v1","addr":"172.21.15.16:6835","nonce":97139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":97139},{"type":"v1","addr":"172.21.15.16:6840","nonce":97139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":97139},{"type":"v1","addr":"172.21.15.16:6837","nonce":97139}]},"public_addr":"172.21.15.16:6833/97139","cluster_addr":"172.21.15.16:6835/97139","heartbeat_back_addr":"172.21.15.16:6840/97139","heartbeat_front_addr":"172.21.15.16:6837/97139","state":["exists","up"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":97135},{"type":"v1","addr":"172.21.15.16:6817","nonce":97135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":97135},{"type":"v1","addr":"172.21.15.16:6819","nonce":97135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":97135},{"type":"v1","addr":"172.21.15.16:6823","nonce":97135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":97135},{"type":"v1","addr":"172.21.15.16:6821","nonce":97135}]},"public_addr":"172.21.15.16:6817/97135","cluster_addr":"172.21.15.16:6819/97135","heartbeat_back_addr":"172.21.15.16:6823/97135","heartbeat_front_addr":"172.21.15.16:6821/97135","state":["exists","up"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":97137},{"type":"v1","addr":"172.21.15.16:6801","nonce":97137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":97137},{"type":"v1","addr":"172.21.15.16:6803","nonce":97137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":97137},{"type":"v1","addr":"172.21.15.16:6807","nonce":97137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":97137},{"type":"v1","addr":"172.21.15.16:6805","nonce":97137}]},"public_addr":"172.21.15.16:6801/97137","cluster_addr":"172.21.15.16:6803/97137","heartbeat_back_addr":"172.21.15.16:6807/97137","heartbeat_front_addr":"172.21.15.16:6805/97137","state":["exists","up"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6848","nonce":97157},{"type":"v1","addr":"172.21.15.16:6851","nonce":97157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6854","nonce":97157},{"type":"v1","addr":"172.21.15.16:6857","nonce":97157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6862","nonce":97157},{"type":"v1","addr":"172.21.15.16:6863","nonce":97157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6860","nonce":97157},{"type":"v1","addr":"172.21.15.16:6861","nonce":97157}]},"public_addr":"172.21.15.16:6851/97157","cluster_addr":"172.21.15.16:6857/97157","heartbeat_back_addr":"172.21.15.16:6863/97157","heartbeat_front_addr":"172.21.15.16:6861/97157","state":["exists","up"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6839","nonce":97145},{"type":"v1","addr":"172.21.15.16:6842","nonce":97145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6844","nonce":97145},{"type":"v1","addr":"172.21.15.16:6846","nonce":97145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6855","nonce":97145},{"type":"v1","addr":"172.21.15.16:6859","nonce":97145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6849","nonce":97145},{"type":"v1","addr":"172.21.15.16:6852","nonce":97145}]},"public_addr":"172.21.15.16:6842/97145","cluster_addr":"172.21.15.16:6846/97145","heartbeat_back_addr":"172.21.15.16:6859/97145","heartbeat_front_addr":"172.21.15.16:6852/97145","state":["exists","up"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6872","nonce":97243},{"type":"v1","addr":"172.21.15.16:6873","nonce":97243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6874","nonce":97243},{"type":"v1","addr":"172.21.15.16:6875","nonce":97243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6878","nonce":97243},{"type":"v1","addr":"172.21.15.16:6879","nonce":97243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6876","nonce":97243},{"type":"v1","addr":"172.21.15.16:6877","nonce":97243}]},"public_addr":"172.21.15.16:6873/97243","cluster_addr":"172.21.15.16:6875/97243","heartbeat_back_addr":"172.21.15.16:6879/97243","heartbeat_front_addr":"172.21.15.16:6877/97243","state":["exists","up"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6864","nonce":97237},{"type":"v1","addr":"172.21.15.16:6865","nonce":97237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6866","nonce":97237},{"type":"v1","addr":"172.21.15.16:6867","nonce":97237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6870","nonce":97237},{"type":"v1","addr":"172.21.15.16:6871","nonce":97237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6868","nonce":97237},{"type":"v1","addr":"172.21.15.16:6869","nonce":97237}]},"public_addr":"172.21.15.16:6865/97237","cluster_addr":"172.21.15.16:6867/97237","heartbeat_back_addr":"172.21.15.16:6871/97237","heartbeat_front_addr":"172.21.15.16:6869/97237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.619173+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.900051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.896545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.850003+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.626569+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.630371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.931585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.898826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.893908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.906245+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:20:20.844 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-01-25T04:20:20.845 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T04:20:21.177 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:21.177 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":19,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:20:21.068784+0000","last_up_change":"2023-01-25T04:20:15.032345+0000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T04:20:18.715898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":97132},{"type":"v1","addr":"172.21.15.16:6809","nonce":97132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":97132},{"type":"v1","addr":"172.21.15.16:6811","nonce":97132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":97132},{"type":"v1","addr":"172.21.15.16:6815","nonce":97132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":97132},{"type":"v1","addr":"172.21.15.16:6813","nonce":97132}]},"public_addr":"172.21.15.16:6809/97132","cluster_addr":"172.21.15.16:6811/97132","heartbeat_back_addr":"172.21.15.16:6815/97132","heartbeat_front_addr":"172.21.15.16:6813/97132","state":["exists","up"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":97131},{"type":"v1","addr":"172.21.15.16:6825","nonce":97131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":97131},{"type":"v1","addr":"172.21.15.16:6827","nonce":97131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":97131},{"type":"v1","addr":"172.21.15.16:6831","nonce":97131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":97131},{"type":"v1","addr":"172.21.15.16:6829","nonce":97131}]},"public_addr":"172.21.15.16:6825/97131","cluster_addr":"172.21.15.16:6827/97131","heartbeat_back_addr":"172.21.15.16:6831/97131","heartbeat_front_addr":"172.21.15.16:6829/97131","state":["exists","up"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6841","nonce":97156},{"type":"v1","addr":"172.21.15.16:6843","nonce":97156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6845","nonce":97156},{"type":"v1","addr":"172.21.15.16:6847","nonce":97156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6856","nonce":97156},{"type":"v1","addr":"172.21.15.16:6858","nonce":97156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6850","nonce":97156},{"type":"v1","addr":"172.21.15.16:6853","nonce":97156}]},"public_addr":"172.21.15.16:6843/97156","cluster_addr":"172.21.15.16:6847/97156","heartbeat_back_addr":"172.21.15.16:6858/97156","heartbeat_front_addr":"172.21.15.16:6853/97156","state":["exists","up"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":97139},{"type":"v1","addr":"172.21.15.16:6833","nonce":97139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":97139},{"type":"v1","addr":"172.21.15.16:6835","nonce":97139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":97139},{"type":"v1","addr":"172.21.15.16:6840","nonce":97139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":97139},{"type":"v1","addr":"172.21.15.16:6837","nonce":97139}]},"public_addr":"172.21.15.16:6833/97139","cluster_addr":"172.21.15.16:6835/97139","heartbeat_back_addr":"172.21.15.16:6840/97139","heartbeat_front_addr":"172.21.15.16:6837/97139","state":["exists","up"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":97135},{"type":"v1","addr":"172.21.15.16:6817","nonce":97135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":97135},{"type":"v1","addr":"172.21.15.16:6819","nonce":97135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":97135},{"type":"v1","addr":"172.21.15.16:6823","nonce":97135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":97135},{"type":"v1","addr":"172.21.15.16:6821","nonce":97135}]},"public_addr":"172.21.15.16:6817/97135","cluster_addr":"172.21.15.16:6819/97135","heartbeat_back_addr":"172.21.15.16:6823/97135","heartbeat_front_addr":"172.21.15.16:6821/97135","state":["exists","up"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":97137},{"type":"v1","addr":"172.21.15.16:6801","nonce":97137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":97137},{"type":"v1","addr":"172.21.15.16:6803","nonce":97137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":97137},{"type":"v1","addr":"172.21.15.16:6807","nonce":97137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":97137},{"type":"v1","addr":"172.21.15.16:6805","nonce":97137}]},"public_addr":"172.21.15.16:6801/97137","cluster_addr":"172.21.15.16:6803/97137","heartbeat_back_addr":"172.21.15.16:6807/97137","heartbeat_front_addr":"172.21.15.16:6805/97137","state":["exists","up"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6848","nonce":97157},{"type":"v1","addr":"172.21.15.16:6851","nonce":97157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6854","nonce":97157},{"type":"v1","addr":"172.21.15.16:6857","nonce":97157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6862","nonce":97157},{"type":"v1","addr":"172.21.15.16:6863","nonce":97157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6860","nonce":97157},{"type":"v1","addr":"172.21.15.16:6861","nonce":97157}]},"public_addr":"172.21.15.16:6851/97157","cluster_addr":"172.21.15.16:6857/97157","heartbeat_back_addr":"172.21.15.16:6863/97157","heartbeat_front_addr":"172.21.15.16:6861/97157","state":["exists","up"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6839","nonce":97145},{"type":"v1","addr":"172.21.15.16:6842","nonce":97145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6844","nonce":97145},{"type":"v1","addr":"172.21.15.16:6846","nonce":97145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6855","nonce":97145},{"type":"v1","addr":"172.21.15.16:6859","nonce":97145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6849","nonce":97145},{"type":"v1","addr":"172.21.15.16:6852","nonce":97145}]},"public_addr":"172.21.15.16:6842/97145","cluster_addr":"172.21.15.16:6846/97145","heartbeat_back_addr":"172.21.15.16:6859/97145","heartbeat_front_addr":"172.21.15.16:6852/97145","state":["exists","up"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6872","nonce":97243},{"type":"v1","addr":"172.21.15.16:6873","nonce":97243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6874","nonce":97243},{"type":"v1","addr":"172.21.15.16:6875","nonce":97243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6878","nonce":97243},{"type":"v1","addr":"172.21.15.16:6879","nonce":97243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6876","nonce":97243},{"type":"v1","addr":"172.21.15.16:6877","nonce":97243}]},"public_addr":"172.21.15.16:6873/97243","cluster_addr":"172.21.15.16:6875/97243","heartbeat_back_addr":"172.21.15.16:6879/97243","heartbeat_front_addr":"172.21.15.16:6877/97243","state":["exists","up"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6864","nonce":97237},{"type":"v1","addr":"172.21.15.16:6865","nonce":97237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6866","nonce":97237},{"type":"v1","addr":"172.21.15.16:6867","nonce":97237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6870","nonce":97237},{"type":"v1","addr":"172.21.15.16:6871","nonce":97237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6868","nonce":97237},{"type":"v1","addr":"172.21.15.16:6869","nonce":97237}]},"public_addr":"172.21.15.16:6865/97237","cluster_addr":"172.21.15.16:6867/97237","heartbeat_back_addr":"172.21.15.16:6871/97237","heartbeat_front_addr":"172.21.15.16:6869/97237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.619173+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.900051+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.896545+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.850003+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.626569+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.630371+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.931585+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.898826+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.893908+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.906245+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:20:21.191 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-01-25T04:20:21.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-01-25T04:20:21.192 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-01-25T04:20:21.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-01-25T04:20:21.193 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-01-25T04:20:21.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-01-25T04:20:21.194 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-01-25T04:20:21.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-01-25T04:20:21.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2023-01-25T04:20:21.196 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2023-01-25T04:20:21.533 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.534 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-01-25T04:20:21.535 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.536 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-01-25T04:20:21.542 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-01-25T04:20:21.545 INFO:teuthology.orchestra.run.smithi016.stdout:64424509444 2023-01-25T04:20:21.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-01-25T04:20:21.561 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.562 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-01-25T04:20:21.565 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2023-01-25T04:20:21.583 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.583 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-01-25T04:20:21.605 INFO:teuthology.orchestra.run.smithi016.stdout:64424509444 2023-01-25T04:20:21.605 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-01-25T04:20:21.643 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.644 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-01-25T04:20:21.658 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:21.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2023-01-25T04:20:22.263 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.268 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.269 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.291 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.0 2023-01-25T04:20:22.291 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.313 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.1 2023-01-25T04:20:22.313 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.317 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.4 2023-01-25T04:20:22.317 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.320 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.322 INFO:teuthology.orchestra.run.smithi016.stdout:64424509444 2023-01-25T04:20:22.334 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.345 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.2 2023-01-25T04:20:22.345 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.8 2023-01-25T04:20:22.347 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.5 2023-01-25T04:20:22.355 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.369 INFO:teuthology.orchestra.run.smithi016.stdout:64424509444 2023-01-25T04:20:22.389 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509444 got 64424509444 for osd.6 2023-01-25T04:20:22.389 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.446 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.461 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.3 2023-01-25T04:20:22.461 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.502 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.502 INFO:teuthology.orchestra.run.smithi016.stdout:64424509443 2023-01-25T04:20:22.517 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.7 2023-01-25T04:20:22.518 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.518 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509443 got 64424509443 for osd.9 2023-01-25T04:20:22.519 DEBUG:teuthology.parallel:result is None 2023-01-25T04:20:22.519 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T04:20:22.519 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:20:22.845 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:22.847 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:20:22.859 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T04:20:21.904270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":50400,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":48640,"kb_avail":1048525600,"statfs":{"total":1073741824000,"available":1073690214400,"internally_reserved":0,"allocated":1720320,"data_stored":533270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":49807360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:21.041785+0000","last_change":"2023-01-25T04:20:20.223322+0000","last_active":"2023-01-25T04:20:21.041785+0000","last_peered":"2023-01-25T04:20:21.041785+0000","last_clean":"2023-01-25T04:20:21.041785+0000","last_became_active":"2023-01-25T04:20:20.223036+0000","last_became_peered":"2023-01-25T04:20:20.223036+0000","last_unstale":"2023-01-25T04:20:21.041785+0000","last_undegraded":"2023-01-25T04:20:21.041785+0000","last_fullsized":"2023-01-25T04:20:21.041785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:36:45.355712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:21.041759+0000","last_change":"2023-01-25T04:20:20.222568+0000","last_active":"2023-01-25T04:20:21.041759+0000","last_peered":"2023-01-25T04:20:21.041759+0000","last_clean":"2023-01-25T04:20:21.041759+0000","last_became_active":"2023-01-25T04:20:20.222342+0000","last_became_peered":"2023-01-25T04:20:20.222342+0000","last_unstale":"2023-01-25T04:20:21.041759+0000","last_undegraded":"2023-01-25T04:20:21.041759+0000","last_fullsized":"2023-01-25T04:20:21.041759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:40:37.263518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.075434+0000","last_change":"2023-01-25T04:20:20.074178+0000","last_active":"2023-01-25T04:20:20.075434+0000","last_peered":"2023-01-25T04:20:20.075434+0000","last_clean":"2023-01-25T04:20:20.075434+0000","last_became_active":"2023-01-25T04:20:20.073892+0000","last_became_peered":"2023-01-25T04:20:20.073892+0000","last_unstale":"2023-01-25T04:20:20.075434+0000","last_undegraded":"2023-01-25T04:20:20.075434+0000","last_fullsized":"2023-01-25T04:20:20.075434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:53:43.255983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.077670+0000","last_change":"2023-01-25T04:20:20.077240+0000","last_active":"2023-01-25T04:20:20.077670+0000","last_peered":"2023-01-25T04:20:20.077670+0000","last_clean":"2023-01-25T04:20:20.077670+0000","last_became_active":"2023-01-25T04:20:20.076933+0000","last_became_peered":"2023-01-25T04:20:20.076933+0000","last_unstale":"2023-01-25T04:20:20.077670+0000","last_undegraded":"2023-01-25T04:20:20.077670+0000","last_fullsized":"2023-01-25T04:20:20.077670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:54:26.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.222634+0000","last_change":"2023-01-25T04:20:20.222634+0000","last_active":"2023-01-25T04:20:20.222634+0000","last_peered":"2023-01-25T04:20:20.222634+0000","last_clean":"2023-01-25T04:20:20.222634+0000","last_became_active":"2023-01-25T04:20:20.222377+0000","last_became_peered":"2023-01-25T04:20:20.222377+0000","last_unstale":"2023-01-25T04:20:20.222634+0000","last_undegraded":"2023-01-25T04:20:20.222634+0000","last_fullsized":"2023-01-25T04:20:20.222634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:54:26.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.222502+0000","last_change":"2023-01-25T04:20:20.222502+0000","last_active":"2023-01-25T04:20:20.222502+0000","last_peered":"2023-01-25T04:20:20.222502+0000","last_clean":"2023-01-25T04:20:20.222502+0000","last_became_active":"2023-01-25T04:20:20.222292+0000","last_became_peered":"2023-01-25T04:20:20.222292+0000","last_unstale":"2023-01-25T04:20:20.222502+0000","last_undegraded":"2023-01-25T04:20:20.222502+0000","last_fullsized":"2023-01-25T04:20:20.222502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:36:45.355712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.078441+0000","last_change":"2023-01-25T04:20:20.078139+0000","last_active":"2023-01-25T04:20:20.078441+0000","last_peered":"2023-01-25T04:20:20.078441+0000","last_clean":"2023-01-25T04:20:20.078441+0000","last_became_active":"2023-01-25T04:20:20.077797+0000","last_became_peered":"2023-01-25T04:20:20.077797+0000","last_unstale":"2023-01-25T04:20:20.078441+0000","last_undegraded":"2023-01-25T04:20:20.078441+0000","last_fullsized":"2023-01-25T04:20:20.078441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:53:43.255983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.074720+0000","last_change":"2023-01-25T04:20:20.074720+0000","last_active":"2023-01-25T04:20:20.074720+0000","last_peered":"2023-01-25T04:20:20.074720+0000","last_clean":"2023-01-25T04:20:20.074720+0000","last_became_active":"2023-01-25T04:20:20.074444+0000","last_became_peered":"2023-01-25T04:20:20.074444+0000","last_unstale":"2023-01-25T04:20:20.074720+0000","last_undegraded":"2023-01-25T04:20:20.074720+0000","last_fullsized":"2023-01-25T04:20:20.074720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:40:37.263518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":8,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3730000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5019999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8079999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0779999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2610000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7690000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.675}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4849999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1070000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5559999999999992}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.069}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.609}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9660000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.654999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.814}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.96}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.702999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4480000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.595000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.387}]}]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:20:22.861 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:20:23.186 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:23.187 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:20:23.199 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-01-25T04:20:21.904270+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":14,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":9,"kb":1048576000,"kb_used":50400,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":48640,"kb_avail":1048525600,"statfs":{"total":1073741824000,"available":1073690214400,"internally_reserved":0,"allocated":1720320,"data_stored":533270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":49807360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":48,"apply_latency_ms":48,"commit_latency_ns":48000000,"apply_latency_ns":48000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:21.041785+0000","last_change":"2023-01-25T04:20:20.223322+0000","last_active":"2023-01-25T04:20:21.041785+0000","last_peered":"2023-01-25T04:20:21.041785+0000","last_clean":"2023-01-25T04:20:21.041785+0000","last_became_active":"2023-01-25T04:20:20.223036+0000","last_became_peered":"2023-01-25T04:20:20.223036+0000","last_unstale":"2023-01-25T04:20:21.041785+0000","last_undegraded":"2023-01-25T04:20:21.041785+0000","last_fullsized":"2023-01-25T04:20:21.041785+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:36:45.355712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:21.041759+0000","last_change":"2023-01-25T04:20:20.222568+0000","last_active":"2023-01-25T04:20:21.041759+0000","last_peered":"2023-01-25T04:20:21.041759+0000","last_clean":"2023-01-25T04:20:21.041759+0000","last_became_active":"2023-01-25T04:20:20.222342+0000","last_became_peered":"2023-01-25T04:20:20.222342+0000","last_unstale":"2023-01-25T04:20:21.041759+0000","last_undegraded":"2023-01-25T04:20:21.041759+0000","last_fullsized":"2023-01-25T04:20:21.041759+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:40:37.263518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9],"acting":[6,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.075434+0000","last_change":"2023-01-25T04:20:20.074178+0000","last_active":"2023-01-25T04:20:20.075434+0000","last_peered":"2023-01-25T04:20:20.075434+0000","last_clean":"2023-01-25T04:20:20.075434+0000","last_became_active":"2023-01-25T04:20:20.073892+0000","last_became_peered":"2023-01-25T04:20:20.073892+0000","last_unstale":"2023-01-25T04:20:20.075434+0000","last_undegraded":"2023-01-25T04:20:20.075434+0000","last_fullsized":"2023-01-25T04:20:20.075434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:53:43.255983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8],"acting":[5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.077670+0000","last_change":"2023-01-25T04:20:20.077240+0000","last_active":"2023-01-25T04:20:20.077670+0000","last_peered":"2023-01-25T04:20:20.077670+0000","last_clean":"2023-01-25T04:20:20.077670+0000","last_became_active":"2023-01-25T04:20:20.076933+0000","last_became_peered":"2023-01-25T04:20:20.076933+0000","last_unstale":"2023-01-25T04:20:20.077670+0000","last_undegraded":"2023-01-25T04:20:20.077670+0000","last_fullsized":"2023-01-25T04:20:20.077670+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:54:26.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0],"acting":[8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.222634+0000","last_change":"2023-01-25T04:20:20.222634+0000","last_active":"2023-01-25T04:20:20.222634+0000","last_peered":"2023-01-25T04:20:20.222634+0000","last_clean":"2023-01-25T04:20:20.222634+0000","last_became_active":"2023-01-25T04:20:20.222377+0000","last_became_peered":"2023-01-25T04:20:20.222377+0000","last_unstale":"2023-01-25T04:20:20.222634+0000","last_undegraded":"2023-01-25T04:20:20.222634+0000","last_fullsized":"2023-01-25T04:20:20.222634+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:54:26.060279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.222502+0000","last_change":"2023-01-25T04:20:20.222502+0000","last_active":"2023-01-25T04:20:20.222502+0000","last_peered":"2023-01-25T04:20:20.222502+0000","last_clean":"2023-01-25T04:20:20.222502+0000","last_became_active":"2023-01-25T04:20:20.222292+0000","last_became_peered":"2023-01-25T04:20:20.222292+0000","last_unstale":"2023-01-25T04:20:20.222502+0000","last_undegraded":"2023-01-25T04:20:20.222502+0000","last_fullsized":"2023-01-25T04:20:20.222502+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T15:36:45.355712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":10,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.078441+0000","last_change":"2023-01-25T04:20:20.078139+0000","last_active":"2023-01-25T04:20:20.078441+0000","last_peered":"2023-01-25T04:20:20.078441+0000","last_clean":"2023-01-25T04:20:20.078441+0000","last_became_active":"2023-01-25T04:20:20.077797+0000","last_became_peered":"2023-01-25T04:20:20.077797+0000","last_unstale":"2023-01-25T04:20:20.078441+0000","last_undegraded":"2023-01-25T04:20:20.078441+0000","last_fullsized":"2023-01-25T04:20:20.078441+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:53:43.255983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":18,"state":"active+clean","last_fresh":"2023-01-25T04:20:20.074720+0000","last_change":"2023-01-25T04:20:20.074720+0000","last_active":"2023-01-25T04:20:20.074720+0000","last_peered":"2023-01-25T04:20:20.074720+0000","last_clean":"2023-01-25T04:20:20.074720+0000","last_became_active":"2023-01-25T04:20:20.074444+0000","last_became_peered":"2023-01-25T04:20:20.074444+0000","last_unstale":"2023-01-25T04:20:20.074720+0000","last_undegraded":"2023-01-25T04:20:20.074720+0000","last_fullsized":"2023-01-25T04:20:20.074720+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:40:37.263518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8],"acting":[4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":8,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]}]},{"osd":7,"up_from":15,"seq":64424509443,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.3730000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.5019999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.8079999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.202}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0779999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.1299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2610000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.7690000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89100000000000001}]}]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33400000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68899999999999995}]}]},{"osd":1,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.675}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29099999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.4849999999999994}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.585}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.1070000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.5559999999999992}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":7.069}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":9.609}]}]},{"osd":0,"up_from":15,"seq":64424509443,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[1,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67100000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65900000000000003}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73199999999999998}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69099999999999995}]}]},{"osd":2,"up_from":15,"seq":64424509444,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9660000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.654999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.814}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.96}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.702999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.4480000000000004}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":15.595000000000001}]}]},{"osd":3,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]}]},{"osd":4,"up_from":15,"seq":64424509443,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.5429999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.494}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.387}]}]},{"osd":5,"up_from":15,"seq":64424509443,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5040,"kb_used_data":168,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":104852560,"statfs":{"total":107374182400,"available":107369021440,"internally_reserved":0,"allocated":172032,"data_stored":53327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:20:23.200 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T04:20:23.200 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-01-25T04:20:23.201 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-01-25T04:20:23.201 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-01-25T04:20:23.568 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:20:23.568 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-01-25T04:20:23.582 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-01-25T04:20:23.582 INFO:teuthology.run_tasks:Running task workunit... 2023-01-25T04:20:23.595 INFO:tasks.workunit:Pulling workunits from ref b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:20:23.596 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-01-25T04:20:23.596 INFO:tasks.workunit:timeout=3h 2023-01-25T04:20:23.597 INFO:tasks.workunit:cleanup=True 2023-01-25T04:20:23.597 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2023-01-25T04:20:23.614 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-01-25T04:20:23.614 INFO:teuthology.orchestra.run.smithi016.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-01-25T04:20:23.615 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -- /home/ubuntu/cephtest/mnt.0 2023-01-25T04:20:23.671 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2023-01-25T04:20:23.672 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2023-01-25T04:20:23.729 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b0d2592529629ab4f86fed5991e556a87dcda563 2023-01-25T04:20:23.785 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-01-25T04:21:04.931 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 99% (11660/11669) Updating files: 100% (11669/11669) Updating files: 100% (11669/11669), done. 2023-01-25T04:21:05.571 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to 'b0d2592529629ab4f86fed5991e556a87dcda563'. 2023-01-25T04:21:05.571 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.571 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-01-25T04:21:05.572 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2023-01-25T04:21:05.572 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2023-01-25T04:21:05.572 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.572 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2023-01-25T04:21:05.572 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2023-01-25T04:21:05.573 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.573 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2023-01-25T04:21:05.573 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.573 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2023-01-25T04:21:05.573 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.574 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2023-01-25T04:21:05.574 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.574 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-01-25T04:21:05.574 INFO:tasks.workunit.client.0.smithi016.stderr: 2023-01-25T04:21:05.574 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at b0d25925296 Merge branch 'wip-rf-unique-formatter' of https://github.com/ronen-fr/ceph into wip-yuri-testing-2023-01-23-0926 2023-01-25T04:21:05.575 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-01-25T04:21:05.643 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-01-25T04:21:05.645 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-25T04:21:05.645 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-01-25T04:21:06.208 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-01-25T04:21:06.261 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-01-25T04:21:06.301 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-01-25T04:21:06.303 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-25T04:21:06.303 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-01-25T04:21:06.340 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-01-25T04:21:06.343 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2023-01-25T04:21:06.344 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-01-25T04:21:06.401 INFO:tasks.workunit:Running workunits matching rados/test_health_warnings.sh on client.0... 2023-01-25T04:21:06.402 INFO:tasks.workunit:Running workunit rados/test_health_warnings.sh... 2023-01-25T04:21:06.402 DEBUG:teuthology.orchestra.run.smithi016:workunit test rados/test_health_warnings.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b0d2592529629ab4f86fed5991e556a87dcda563 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_health_warnings.sh 2023-01-25T04:21:06.462 INFO:tasks.workunit.client.0.smithi016.stderr:+ crushtool -o crushmap --build --num_osds 10 host straw 2 rack straw 2 row straw 2 root straw 0 2023-01-25T04:21:06.478 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd setcrushmap -i crushmap 2023-01-25T04:21:08.176 INFO:tasks.workunit.client.0.smithi016.stderr:6 2023-01-25T04:21:08.190 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd tree 2023-01-25T04:21:08.634 INFO:tasks.workunit.client.0.smithi016.stdout:ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF 2023-01-25T04:21:08.635 INFO:tasks.workunit.client.0.smithi016.stdout:-11 10.00000 root root 2023-01-25T04:21:08.635 INFO:tasks.workunit.client.0.smithi016.stdout: -9 8.00000 row row0 2023-01-25T04:21:08.635 INFO:tasks.workunit.client.0.smithi016.stdout: -6 4.00000 rack rack0 2023-01-25T04:21:08.635 INFO:tasks.workunit.client.0.smithi016.stdout: -1 2.00000 host host0 2023-01-25T04:21:08.635 INFO:tasks.workunit.client.0.smithi016.stdout: 0 1.00000 osd.0 up 1.00000 1.00000 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: 1 1.00000 osd.1 up 1.00000 1.00000 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: -2 2.00000 host host1 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: 2 1.00000 osd.2 up 1.00000 1.00000 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: 3 1.00000 osd.3 up 1.00000 1.00000 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: -7 4.00000 rack rack1 2023-01-25T04:21:08.636 INFO:tasks.workunit.client.0.smithi016.stdout: -3 2.00000 host host2 2023-01-25T04:21:08.637 INFO:tasks.workunit.client.0.smithi016.stdout: 4 1.00000 osd.4 up 1.00000 1.00000 2023-01-25T04:21:08.637 INFO:tasks.workunit.client.0.smithi016.stdout: 5 1.00000 osd.5 up 1.00000 1.00000 2023-01-25T04:21:08.637 INFO:tasks.workunit.client.0.smithi016.stdout: -4 2.00000 host host3 2023-01-25T04:21:08.637 INFO:tasks.workunit.client.0.smithi016.stdout: 6 1.00000 osd.6 up 1.00000 1.00000 2023-01-25T04:21:08.638 INFO:tasks.workunit.client.0.smithi016.stdout: 7 1.00000 osd.7 up 1.00000 1.00000 2023-01-25T04:21:08.638 INFO:tasks.workunit.client.0.smithi016.stdout:-10 2.00000 row row1 2023-01-25T04:21:08.638 INFO:tasks.workunit.client.0.smithi016.stdout: -8 2.00000 rack rack2 2023-01-25T04:21:08.638 INFO:tasks.workunit.client.0.smithi016.stdout: -5 2.00000 host host4 2023-01-25T04:21:08.639 INFO:tasks.workunit.client.0.smithi016.stdout: 8 1.00000 osd.8 up 1.00000 1.00000 2023-01-25T04:21:08.639 INFO:tasks.workunit.client.0.smithi016.stdout: 9 1.00000 osd.9 up 1.00000 1.00000 2023-01-25T04:21:08.647 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph tell 'osd.*' injectargs --osd_max_markdown_count 1024 --osd_max_markdown_period 1 2023-01-25T04:21:08.787 INFO:tasks.workunit.client.0.smithi016.stdout:osd.0: {} 2023-01-25T04:21:08.821 INFO:tasks.workunit.client.0.smithi016.stdout:osd.1: {} 2023-01-25T04:21:08.855 INFO:tasks.workunit.client.0.smithi016.stdout:osd.2: {} 2023-01-25T04:21:08.889 INFO:tasks.workunit.client.0.smithi016.stdout:osd.3: {} 2023-01-25T04:21:08.923 INFO:tasks.workunit.client.0.smithi016.stdout:osd.4: {} 2023-01-25T04:21:08.956 INFO:tasks.workunit.client.0.smithi016.stdout:osd.5: {} 2023-01-25T04:21:08.990 INFO:tasks.workunit.client.0.smithi016.stdout:osd.6: {} 2023-01-25T04:21:09.024 INFO:tasks.workunit.client.0.smithi016.stdout:osd.7: {} 2023-01-25T04:21:09.057 INFO:tasks.workunit.client.0.smithi016.stdout:osd.8: {} 2023-01-25T04:21:09.090 INFO:tasks.workunit.client.0.smithi016.stdout:osd.9: {} 2023-01-25T04:21:09.101 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd set noout 2023-01-25T04:21:11.174 INFO:tasks.workunit.client.0.smithi016.stderr:noout is set 2023-01-25T04:21:11.186 INFO:tasks.workunit.client.0.smithi016.stderr:+ test_mark_two_osds_same_host_down 2023-01-25T04:21:11.186 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd set noup 2023-01-25T04:21:13.178 INFO:tasks.workunit.client.0.smithi016.stderr:noup is set 2023-01-25T04:21:13.189 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd down osd.0 osd.1 2023-01-25T04:21:13.511 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:13.509+0000 7fba60d41700 -1 mon.a@0(leader).osd e25 definitely_dead 0 2023-01-25T04:21:14.324 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:14.322+0000 7fba60d41700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2023-01-25T04:21:14.326 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 is already down. osd.1 is already down. 2023-01-25T04:21:14.336 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:14.920 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:14.920 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-25T04:21:14.920 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-25T04:21:14.920 INFO:tasks.workunit.client.0.smithi016.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:14.921 INFO:tasks.workunit.client.0.smithi016.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:14.921 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-25T04:21:14.921 INFO:tasks.workunit.client.0.smithi016.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-25T04:21:14.933 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:14.933 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 host' 2023-01-25T04:21:15.425 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:15.423+0000 7f8f19cc4700 -1 osd.1 27 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:15.472 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:15.472 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:15.473 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '2 osds' 2023-01-25T04:21:15.629 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:15.628+0000 7f8c392f4700 -1 osd.0 27 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:16.003 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:16.004 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:16.004 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.0 2023-01-25T04:21:16.184 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:16.182+0000 7f8f0c093700 -1 osd.1 28 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:16.184 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:16.182+0000 7f8c2aec2700 -1 osd.0 28 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:16.608 INFO:tasks.workunit.client.0.smithi016.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:16.609 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:16.609 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.1 2023-01-25T04:21:17.208 INFO:tasks.workunit.client.0.smithi016.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:17.209 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd unset noup 2023-01-25T04:21:18.189 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:18.186+0000 7f8c32ed2700 -1 osd.0 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:18.190 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:18.187+0000 7f8f140a3700 -1 osd.1 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:19.194 INFO:tasks.workunit.client.0.smithi016.stderr:noup is unset 2023-01-25T04:21:19.208 INFO:tasks.workunit.client.0.smithi016.stderr:+ wait_for_healthy 2023-01-25T04:21:19.209 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:19.209 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep down 2023-01-25T04:21:19.751 INFO:tasks.workunit.client.0.smithi016.stderr:+ test_mark_two_osds_same_rack_down 2023-01-25T04:21:19.751 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd set noup 2023-01-25T04:21:21.200 INFO:tasks.workunit.client.0.smithi016.stderr:noup is set 2023-01-25T04:21:21.213 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd down osd.8 osd.9 2023-01-25T04:21:21.539 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:21.538+0000 7fba60d41700 -1 mon.a@0(leader).osd e32 definitely_dead 0 2023-01-25T04:21:22.343 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:22.341+0000 7fba60d41700 -1 mon.a@0(leader).osd e33 definitely_dead 0 2023-01-25T04:21:22.346 INFO:tasks.workunit.client.0.smithi016.stderr:osd.8 is already down. osd.9 is already down. 2023-01-25T04:21:22.356 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:22.944 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-25T04:21:22.945 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-25T04:21:22.945 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-25T04:21:22.945 INFO:tasks.workunit.client.0.smithi016.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-25T04:21:22.945 INFO:tasks.workunit.client.0.smithi016.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-25T04:21:22.946 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-25T04:21:22.946 INFO:tasks.workunit.client.0.smithi016.stdout: host host4 (root=root,row=row1,rack=rack2) (2 osds) is down 2023-01-25T04:21:22.946 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_RACK_DOWN: 1 rack (2 osds) down 2023-01-25T04:21:22.946 INFO:tasks.workunit.client.0.smithi016.stdout: rack rack2 (root=root,row=row1) (2 osds) is down 2023-01-25T04:21:22.946 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_ROW_DOWN: 1 row (2 osds) down 2023-01-25T04:21:22.947 INFO:tasks.workunit.client.0.smithi016.stdout: row row1 (root=root) (2 osds) is down 2023-01-25T04:21:22.956 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:22.957 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 host' 2023-01-25T04:21:23.488 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-25T04:21:23.489 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:23.489 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 rack' 2023-01-25T04:21:24.019 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-25T04:21:24.019 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:24.020 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 row' 2023-01-25T04:21:24.158 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:21:24.156+0000 7f8d89f01700 -1 osd.9 33 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:24.203 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:21:24.201+0000 7f8d7cad1700 -1 osd.9 34 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:24.554 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-25T04:21:24.554 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:24.555 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '2 osds' 2023-01-25T04:21:25.085 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down; 1 rack (2 osds) down; 1 row (2 osds) down 2023-01-25T04:21:25.086 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:25.087 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.8 2023-01-25T04:21:25.205 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:21:25.203+0000 7f8d7cad1700 -1 osd.9 35 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:25.415 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:25.414+0000 7f986c003700 -1 osd.8 35 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:25.689 INFO:tasks.workunit.client.0.smithi016.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-25T04:21:25.690 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:25.690 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.9 2023-01-25T04:21:26.297 INFO:tasks.workunit.client.0.smithi016.stdout: osd.9 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-25T04:21:26.297 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd unset noup 2023-01-25T04:21:27.210 INFO:tasks.ceph.osd.9.smithi016.stderr:2023-01-25T04:21:27.207+0000 7f8d84ae1700 -1 osd.9 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:27.210 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:27.208+0000 7f9866be3700 -1 osd.8 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:28.221 INFO:tasks.workunit.client.0.smithi016.stderr:noup is unset 2023-01-25T04:21:28.233 INFO:tasks.workunit.client.0.smithi016.stderr:+ wait_for_healthy 2023-01-25T04:21:28.233 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:28.233 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep down 2023-01-25T04:21:28.771 INFO:tasks.workunit.client.0.smithi016.stderr:+ test_mark_all_but_last_osds_down 2023-01-25T04:21:28.771 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd set noup 2023-01-25T04:21:30.225 INFO:tasks.workunit.client.0.smithi016.stderr:noup is set 2023-01-25T04:21:30.239 INFO:tasks.workunit.client.0.smithi016.stderr:++ ceph osd ls 2023-01-25T04:21:30.239 INFO:tasks.workunit.client.0.smithi016.stderr:++ sed '$d' 2023-01-25T04:21:30.692 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd down 0 1 2 3 4 5 6 7 8 2023-01-25T04:21:31.016 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:31.016+0000 7fba60d41700 -1 mon.a@0(leader).osd e39 definitely_dead 0 2023-01-25T04:21:32.146 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:32.146+0000 7fba60d41700 -1 mon.a@0(leader).osd e40 definitely_dead 0 2023-01-25T04:21:32.149 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 is already down. osd.1 is already down. osd.2 is already down. osd.3 is already down. osd.4 is already down. osd.5 is already down. osd.6 is already down. osd.7 is already down. osd.8 is already down. 2023-01-25T04:21:32.159 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:32.692 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:21:32.691+0000 7fa90418b700 -1 osd.7 40 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:32.747 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-25T04:21:32.747 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-25T04:21:32.747 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_DOWN: 9 osds down 2023-01-25T04:21:32.748 INFO:tasks.workunit.client.0.smithi016.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:32.748 INFO:tasks.workunit.client.0.smithi016.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:32.748 INFO:tasks.workunit.client.0.smithi016.stdout: osd.2 (root=root,row=row0,rack=rack0,host=host1) is down 2023-01-25T04:21:32.748 INFO:tasks.workunit.client.0.smithi016.stdout: osd.3 (root=root,row=row0,rack=rack0,host=host1) is down 2023-01-25T04:21:32.748 INFO:tasks.workunit.client.0.smithi016.stdout: osd.4 (root=root,row=row0,rack=rack1,host=host2) is down 2023-01-25T04:21:32.749 INFO:tasks.workunit.client.0.smithi016.stdout: osd.5 (root=root,row=row0,rack=rack1,host=host2) is down 2023-01-25T04:21:32.749 INFO:tasks.workunit.client.0.smithi016.stdout: osd.6 (root=root,row=row0,rack=rack1,host=host3) is down 2023-01-25T04:21:32.749 INFO:tasks.workunit.client.0.smithi016.stdout: osd.7 (root=root,row=row0,rack=rack1,host=host3) is down 2023-01-25T04:21:32.749 INFO:tasks.workunit.client.0.smithi016.stdout: osd.8 (root=root,row=row1,rack=rack2,host=host4) is down 2023-01-25T04:21:32.749 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_HOST_DOWN: 4 hosts (8 osds) down 2023-01-25T04:21:32.750 INFO:tasks.workunit.client.0.smithi016.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-25T04:21:32.750 INFO:tasks.workunit.client.0.smithi016.stdout: host host1 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-25T04:21:32.750 INFO:tasks.workunit.client.0.smithi016.stdout: host host2 (root=root,row=row0,rack=rack1) (2 osds) is down 2023-01-25T04:21:32.750 INFO:tasks.workunit.client.0.smithi016.stdout: host host3 (root=root,row=row0,rack=rack1) (2 osds) is down 2023-01-25T04:21:32.750 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_RACK_DOWN: 2 racks (8 osds) down 2023-01-25T04:21:32.751 INFO:tasks.workunit.client.0.smithi016.stdout: rack rack0 (root=root,row=row0) (4 osds) is down 2023-01-25T04:21:32.751 INFO:tasks.workunit.client.0.smithi016.stdout: rack rack1 (root=root,row=row0) (4 osds) is down 2023-01-25T04:21:32.751 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_ROW_DOWN: 1 row (8 osds) down 2023-01-25T04:21:32.751 INFO:tasks.workunit.client.0.smithi016.stdout: row row0 (root=root) (8 osds) is down 2023-01-25T04:21:32.760 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:32.760 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 row' 2023-01-25T04:21:33.003 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:21:33.002+0000 7fa8f655a700 -1 osd.7 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:33.060 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:21:33.060+0000 7fe5b7c59700 -1 osd.2 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:33.305 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-25T04:21:33.305 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:33.305 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '2 racks' 2023-01-25T04:21:33.439 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:33.439+0000 7f986b802700 -1 osd.8 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:33.610 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:33.609+0000 7f8f194c3700 -1 osd.1 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:33.698 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:21:33.698+0000 7f410ce26700 -1 osd.5 41 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:33.836 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-25T04:21:33.837 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:33.837 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '4 hosts' 2023-01-25T04:21:34.004 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:21:34.003+0000 7f40fe9f4700 -1 osd.5 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.004 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:21:34.003+0000 7fe5aa028700 -1 osd.2 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.005 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:21:34.003+0000 7fa8f655a700 -1 osd.7 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.005 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:34.004+0000 7f8f0c093700 -1 osd.1 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.005 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:34.003+0000 7f985ebd3700 -1 osd.8 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.058 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:21:34.058+0000 7f5e63568700 -1 osd.6 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.370 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-25T04:21:34.371 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:34.371 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '9 osds' 2023-01-25T04:21:34.500 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:34.500+0000 7f8c38af3700 -1 osd.0 42 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:34.901 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 9 osds down; 4 hosts (8 osds) down; 2 racks (8 osds) down; 1 row (8 osds) down 2023-01-25T04:21:34.901 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd unset noup 2023-01-25T04:21:35.004 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:35.004+0000 7f8c2aec2700 -1 osd.0 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.005 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:21:35.004+0000 7fa8f655a700 -1 osd.7 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.005 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:35.004+0000 7f985ebd3700 -1 osd.8 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.006 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:21:35.004+0000 7f40fe9f4700 -1 osd.5 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.006 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:21:35.005+0000 7f5e55937700 -1 osd.6 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.006 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:35.005+0000 7f8f0c093700 -1 osd.1 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.007 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:21:35.006+0000 7fe5aa028700 -1 osd.2 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.021 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:21:35.021+0000 7f7041653700 -1 osd.3 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:35.515 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:21:35.514+0000 7fba5b998700 -1 osd.4 43 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:36.006 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:36.006+0000 7f8c32ed2700 -1 osd.0 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.007 INFO:tasks.ceph.osd.2.smithi016.stderr:2023-01-25T04:21:36.006+0000 7fe5b2038700 -1 osd.2 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.008 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:36.006+0000 7f8f140a3700 -1 osd.1 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.008 INFO:tasks.ceph.osd.7.smithi016.stderr:2023-01-25T04:21:36.006+0000 7fa8fe56a700 -1 osd.7 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.011 INFO:tasks.ceph.osd.5.smithi016.stderr:2023-01-25T04:21:36.007+0000 7f4106a04700 -1 osd.5 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.012 INFO:tasks.ceph.osd.3.smithi016.stderr:2023-01-25T04:21:36.007+0000 7f703ba32700 -1 osd.3 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.012 INFO:tasks.ceph.osd.6.smithi016.stderr:2023-01-25T04:21:36.008+0000 7f5e5d947700 -1 osd.6 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.012 INFO:tasks.ceph.osd.8.smithi016.stderr:2023-01-25T04:21:36.008+0000 7f9866be3700 -1 osd.8 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:36.013 INFO:tasks.ceph.osd.4.smithi016.stderr:2023-01-25T04:21:36.009+0000 7fba55576700 -1 osd.4 44 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:37.034 INFO:tasks.workunit.client.0.smithi016.stderr:noup is unset 2023-01-25T04:21:37.047 INFO:tasks.workunit.client.0.smithi016.stderr:+ wait_for_healthy 2023-01-25T04:21:37.047 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:37.047 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep down 2023-01-25T04:21:37.581 INFO:tasks.workunit.client.0.smithi016.stderr:+ test_mark_two_osds_same_host_down_with_classes 2023-01-25T04:21:37.581 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd set noup 2023-01-25T04:21:39.012 INFO:tasks.workunit.client.0.smithi016.stderr:noup is set 2023-01-25T04:21:39.024 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd crush set-device-class ssd osd.0 osd.2 osd.4 osd.6 osd.8 2023-01-25T04:21:41.016 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 already set to class ssd. set-device-class item id 0 name 'osd.0' device_class 'ssd': no change. osd.2 already set to class ssd. set-device-class item id 2 name 'osd.2' device_class 'ssd': no change. osd.4 already set to class ssd. set-device-class item id 4 name 'osd.4' device_class 'ssd': no change. osd.6 already set to class ssd. set-device-class item id 6 name 'osd.6' device_class 'ssd': no change. osd.8 already set to class ssd. set-device-class item id 8 name 'osd.8' device_class 'ssd': no change. set osd(s) to class 'ssd' 2023-01-25T04:21:41.029 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd crush set-device-class hdd osd.1 osd.3 osd.5 osd.7 osd.9 2023-01-25T04:21:43.023 INFO:tasks.workunit.client.0.smithi016.stderr:osd.1 already set to class hdd. set-device-class item id 1 name 'osd.1' device_class 'hdd': no change. osd.3 already set to class hdd. set-device-class item id 3 name 'osd.3' device_class 'hdd': no change. osd.5 already set to class hdd. set-device-class item id 5 name 'osd.5' device_class 'hdd': no change. osd.7 already set to class hdd. set-device-class item id 7 name 'osd.7' device_class 'hdd': no change. osd.9 already set to class hdd. set-device-class item id 9 name 'osd.9' device_class 'hdd': no change. set osd(s) to class 'hdd' 2023-01-25T04:21:43.035 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd down osd.0 osd.1 2023-01-25T04:21:43.362 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:43.361+0000 7fba60d41700 -1 mon.a@0(leader).osd e51 definitely_dead 0 2023-01-25T04:21:44.171 INFO:tasks.ceph.mon.a.smithi016.stderr:2023-01-25T04:21:44.170+0000 7fba60d41700 -1 mon.a@0(leader).osd e52 definitely_dead 0 2023-01-25T04:21:44.174 INFO:tasks.workunit.client.0.smithi016.stderr:osd.0 is already down. osd.1 is already down. 2023-01-25T04:21:44.184 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:44.621 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:44.621+0000 7f8c392f4700 -1 osd.0 52 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:44.778 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:44.778 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSDMAP_FLAGS: noup,noout flag(s) set 2023-01-25T04:21:44.778 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_DOWN: 2 osds down 2023-01-25T04:21:44.778 INFO:tasks.workunit.client.0.smithi016.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:44.779 INFO:tasks.workunit.client.0.smithi016.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:44.779 INFO:tasks.workunit.client.0.smithi016.stdout:[WRN] OSD_HOST_DOWN: 1 host (2 osds) down 2023-01-25T04:21:44.779 INFO:tasks.workunit.client.0.smithi016.stdout: host host0 (root=root,row=row0,rack=rack0) (2 osds) is down 2023-01-25T04:21:44.791 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:44.791 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '1 host' 2023-01-25T04:21:45.024 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:45.023+0000 7f8c2aec2700 -1 osd.0 53 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:45.325 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:45.326 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:45.326 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep '2 osds' 2023-01-25T04:21:45.855 INFO:tasks.workunit.client.0.smithi016.stdout:HEALTH_WARN noup,noout flag(s) set; 2 osds down; 1 host (2 osds) down 2023-01-25T04:21:45.856 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:45.856 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.0 2023-01-25T04:21:46.029 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:46.027+0000 7f8c2aec2700 -1 osd.0 54 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:46.459 INFO:tasks.workunit.client.0.smithi016.stdout: osd.0 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:46.460 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health detail 2023-01-25T04:21:46.460 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep osd.1 2023-01-25T04:21:47.031 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:47.030+0000 7f8c2aec2700 -1 osd.0 55 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:47.068 INFO:tasks.workunit.client.0.smithi016.stdout: osd.1 (root=root,row=row0,rack=rack0,host=host0) is down 2023-01-25T04:21:47.068 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph osd unset noup 2023-01-25T04:21:47.403 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:47.402+0000 7f8f19cc4700 -1 osd.1 55 osdmap NOUP flag is set, waiting for it to clear 2023-01-25T04:21:48.035 INFO:tasks.ceph.osd.1.smithi016.stderr:2023-01-25T04:21:48.033+0000 7f8f140a3700 -1 osd.1 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:48.036 INFO:tasks.ceph.osd.0.smithi016.stderr:2023-01-25T04:21:48.033+0000 7f8c32ed2700 -1 osd.0 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-01-25T04:21:49.041 INFO:tasks.workunit.client.0.smithi016.stderr:noup is unset 2023-01-25T04:21:49.053 INFO:tasks.workunit.client.0.smithi016.stderr:+ wait_for_healthy 2023-01-25T04:21:49.054 INFO:tasks.workunit.client.0.smithi016.stderr:+ ceph health 2023-01-25T04:21:49.054 INFO:tasks.workunit.client.0.smithi016.stderr:+ grep down 2023-01-25T04:21:49.602 INFO:tasks.workunit.client.0.smithi016.stderr:+ exit 0 2023-01-25T04:21:49.603 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-01-25T04:21:49.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-01-25T04:21:49.670 INFO:tasks.workunit:Stopping ['rados/test_health_warnings.sh'] on client.0... 2023-01-25T04:21:49.670 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-01-25T04:21:49.970 DEBUG:teuthology.parallel:result is None 2023-01-25T04:21:49.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2023-01-25T04:21:49.998 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2023-01-25T04:21:49.998 DEBUG:teuthology.orchestra.run.smithi016:> rmdir -- /home/ubuntu/cephtest/mnt.0 2023-01-25T04:21:50.066 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2023-01-25T04:21:50.067 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-01-25T04:21:50.076 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-01-25T04:21:50.077 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-01-25T04:21:50.435 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:50.435 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":58,"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","created":"2023-01-25T04:20:06.017373+0000","modified":"2023-01-25T04:21:50.037434+0000","last_up_change":"2023-01-25T04:21:49.032643+0000","last_in_change":"2023-01-25T04:20:09.387970+0000","flags":"noout,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5800064,"flags_set":["noout","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":10,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":10,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-01-25T04:20:18.715898+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"843bf900-0cea-4573-b26b-c2fc7fbb0dfc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":56,"up_from":57,"up_thru":33,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":97132},{"type":"v1","addr":"172.21.15.16:6809","nonce":97132}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3097132},{"type":"v1","addr":"172.21.15.16:6819","nonce":3097132}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":97132},{"type":"v1","addr":"172.21.15.16:6815","nonce":97132}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":97132},{"type":"v1","addr":"172.21.15.16:6813","nonce":97132}]},"public_addr":"172.21.15.16:6809/97132","cluster_addr":"172.21.15.16:6819/3097132","heartbeat_back_addr":"172.21.15.16:6815/97132","heartbeat_front_addr":"172.21.15.16:6813/97132","state":["exists","up"]},{"osd":1,"uuid":"20016226-4430-4bf5-891a-4ed2415659af","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":56,"up_from":57,"up_thru":57,"down_at":52,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":97131},{"type":"v1","addr":"172.21.15.16:6825","nonce":97131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6854","nonce":3097131},{"type":"v1","addr":"172.21.15.16:6857","nonce":3097131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":97131},{"type":"v1","addr":"172.21.15.16:6831","nonce":97131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":97131},{"type":"v1","addr":"172.21.15.16:6829","nonce":97131}]},"public_addr":"172.21.15.16:6825/97131","cluster_addr":"172.21.15.16:6857/3097131","heartbeat_back_addr":"172.21.15.16:6831/97131","heartbeat_front_addr":"172.21.15.16:6829/97131","state":["exists","up"]},{"osd":2,"uuid":"1c73c582-5f9a-4f7b-9631-a6dafe254a79","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":0,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6841","nonce":97156},{"type":"v1","addr":"172.21.15.16:6843","nonce":97156}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6844","nonce":1097156},{"type":"v1","addr":"172.21.15.16:6846","nonce":1097156}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6856","nonce":97156},{"type":"v1","addr":"172.21.15.16:6858","nonce":97156}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6850","nonce":97156},{"type":"v1","addr":"172.21.15.16:6853","nonce":97156}]},"public_addr":"172.21.15.16:6843/97156","cluster_addr":"172.21.15.16:6846/1097156","heartbeat_back_addr":"172.21.15.16:6858/97156","heartbeat_front_addr":"172.21.15.16:6853/97156","state":["exists","up"]},{"osd":3,"uuid":"4c0614b6-a46d-4428-8c8c-e87eec7fa624","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":57,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":97139},{"type":"v1","addr":"172.21.15.16:6833","nonce":97139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":1097139},{"type":"v1","addr":"172.21.15.16:6827","nonce":1097139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":97139},{"type":"v1","addr":"172.21.15.16:6840","nonce":97139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":97139},{"type":"v1","addr":"172.21.15.16:6837","nonce":97139}]},"public_addr":"172.21.15.16:6833/97139","cluster_addr":"172.21.15.16:6827/1097139","heartbeat_back_addr":"172.21.15.16:6840/97139","heartbeat_front_addr":"172.21.15.16:6837/97139","state":["exists","up"]},{"osd":4,"uuid":"64fba726-cab2-42b8-81d8-c57bc425dd9c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":57,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":97135},{"type":"v1","addr":"172.21.15.16:6817","nonce":97135}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":1097135},{"type":"v1","addr":"172.21.15.16:6835","nonce":1097135}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":97135},{"type":"v1","addr":"172.21.15.16:6823","nonce":97135}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":97135},{"type":"v1","addr":"172.21.15.16:6821","nonce":97135}]},"public_addr":"172.21.15.16:6817/97135","cluster_addr":"172.21.15.16:6835/1097135","heartbeat_back_addr":"172.21.15.16:6823/97135","heartbeat_front_addr":"172.21.15.16:6821/97135","state":["exists","up"]},{"osd":5,"uuid":"01cbab51-a662-4e35-ae3a-9902a3109369","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":45,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":97137},{"type":"v1","addr":"172.21.15.16:6801","nonce":97137}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6882","nonce":1097137},{"type":"v1","addr":"172.21.15.16:6883","nonce":1097137}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":97137},{"type":"v1","addr":"172.21.15.16:6807","nonce":97137}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":97137},{"type":"v1","addr":"172.21.15.16:6805","nonce":97137}]},"public_addr":"172.21.15.16:6801/97137","cluster_addr":"172.21.15.16:6883/1097137","heartbeat_back_addr":"172.21.15.16:6807/97137","heartbeat_front_addr":"172.21.15.16:6805/97137","state":["exists","up"]},{"osd":6,"uuid":"b0337d52-03c4-4788-b488-3773e40fa420","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":52,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6848","nonce":97157},{"type":"v1","addr":"172.21.15.16:6851","nonce":97157}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1097157},{"type":"v1","addr":"172.21.15.16:6803","nonce":1097157}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6862","nonce":97157},{"type":"v1","addr":"172.21.15.16:6863","nonce":97157}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6860","nonce":97157},{"type":"v1","addr":"172.21.15.16:6861","nonce":97157}]},"public_addr":"172.21.15.16:6851/97157","cluster_addr":"172.21.15.16:6803/1097157","heartbeat_back_addr":"172.21.15.16:6863/97157","heartbeat_front_addr":"172.21.15.16:6861/97157","state":["exists","up"]},{"osd":7,"uuid":"35b039dc-2c63-46c0-b7cc-2ccca15a0866","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":45,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6839","nonce":97145},{"type":"v1","addr":"172.21.15.16:6842","nonce":97145}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6874","nonce":1097145},{"type":"v1","addr":"172.21.15.16:6875","nonce":1097145}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6855","nonce":97145},{"type":"v1","addr":"172.21.15.16:6859","nonce":97145}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6849","nonce":97145},{"type":"v1","addr":"172.21.15.16:6852","nonce":97145}]},"public_addr":"172.21.15.16:6842/97145","cluster_addr":"172.21.15.16:6875/1097145","heartbeat_back_addr":"172.21.15.16:6859/97145","heartbeat_front_addr":"172.21.15.16:6852/97145","state":["exists","up"]},{"osd":8,"uuid":"8d0279b3-fbec-4a30-aaeb-a64c4ad211f1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":44,"up_from":45,"up_thru":17,"down_at":40,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6872","nonce":97243},{"type":"v1","addr":"172.21.15.16:6873","nonce":97243}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6845","nonce":2097243},{"type":"v1","addr":"172.21.15.16:6847","nonce":2097243}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6878","nonce":97243},{"type":"v1","addr":"172.21.15.16:6879","nonce":97243}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6876","nonce":97243},{"type":"v1","addr":"172.21.15.16:6877","nonce":97243}]},"public_addr":"172.21.15.16:6873/97243","cluster_addr":"172.21.15.16:6847/2097243","heartbeat_back_addr":"172.21.15.16:6879/97243","heartbeat_front_addr":"172.21.15.16:6877/97243","state":["exists","up"]},{"osd":9,"uuid":"e9a272ed-ec7e-4307-9130-5249fd3fbc16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":15,"last_clean_end":36,"up_from":37,"up_thru":57,"down_at":33,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6864","nonce":97237},{"type":"v1","addr":"172.21.15.16:6865","nonce":97237}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":1097237},{"type":"v1","addr":"172.21.15.16:6811","nonce":1097237}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6870","nonce":97237},{"type":"v1","addr":"172.21.15.16:6871","nonce":97237}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6868","nonce":97237},{"type":"v1","addr":"172.21.15.16:6869","nonce":97237}]},"public_addr":"172.21.15.16:6865/97237","cluster_addr":"172.21.15.16:6811/1097237","heartbeat_back_addr":"172.21.15.16:6871/97237","heartbeat_front_addr":"172.21.15.16:6869/97237","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"2023-01-25T04:21:44.019547+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.619173+0000","dead_epoch":52},{"osd":1,"down_stamp":"2023-01-25T04:21:44.019547+0000","laggy_probability":0.65700000524520874,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.900051+0000","dead_epoch":54},{"osd":2,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.896545+0000","dead_epoch":40},{"osd":3,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.850003+0000","dead_epoch":42},{"osd":4,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.626569+0000","dead_epoch":43},{"osd":5,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.630371+0000","dead_epoch":41},{"osd":6,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.931585+0000","dead_epoch":41},{"osd":7,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.898826+0000","dead_epoch":40},{"osd":8,"down_stamp":"2023-01-25T04:21:32.000270+0000","laggy_probability":0.50999999046325684,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.893908+0000","dead_epoch":40},{"osd":9,"down_stamp":"2023-01-25T04:21:22.195974+0000","laggy_probability":0.30000001192092896,"laggy_interval":1,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-01-25T04:20:13.906245+0000","dead_epoch":33}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-01-25T04:21:50.449 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-01-25T04:21:50.449 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:50.776 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:50.778 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:50.789 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-01-25T04:21:50.039442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":57628,"kb_used_data":5772,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":1048518372,"statfs":{"total":1073741824000,"available":1073682812928,"internally_reserved":0,"allocated":5910528,"data_stored":2770956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702632+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:44.702632+0000","last_peered":"2023-01-25T04:21:44.702632+0000","last_clean":"2023-01-25T04:21:44.702632+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:44.702632+0000","last_undegraded":"2023-01-25T04:21:44.702632+0000","last_fullsized":"2023-01-25T04:21:44.702632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.023808+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:44.023808+0000","last_peered":"2023-01-25T04:21:44.023808+0000","last_clean":"2023-01-25T04:21:44.023808+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:44.023808+0000","last_undegraded":"2023-01-25T04:21:44.023808+0000","last_fullsized":"2023-01-25T04:21:44.023808+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.017542+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:42.017542+0000","last_peered":"2023-01-25T04:21:42.017542+0000","last_clean":"2023-01-25T04:21:42.017542+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:42.017542+0000","last_undegraded":"2023-01-25T04:21:42.017542+0000","last_fullsized":"2023-01-25T04:21:42.017542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702378+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:44.702378+0000","last_peered":"2023-01-25T04:21:44.702378+0000","last_clean":"2023-01-25T04:21:44.702378+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:44.702378+0000","last_undegraded":"2023-01-25T04:21:44.702378+0000","last_fullsized":"2023-01-25T04:21:44.702378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":58,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.797363+0000","last_change":"2023-01-25T04:21:38.011605+0000","last_active":"2023-01-25T04:21:42.797363+0000","last_peered":"2023-01-25T04:21:42.797363+0000","last_clean":"2023-01-25T04:21:42.797363+0000","last_became_active":"2023-01-25T04:21:38.011375+0000","last_became_peered":"2023-01-25T04:21:38.011375+0000","last_unstale":"2023-01-25T04:21:42.797363+0000","last_undegraded":"2023-01-25T04:21:42.797363+0000","last_fullsized":"2023-01-25T04:21:42.797363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:44:46.761850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+undersized+wait","last_fresh":"2023-01-25T04:21:45.026434+0000","last_change":"2023-01-25T04:21:45.026371+0000","last_active":"2023-01-25T04:21:45.026434+0000","last_peered":"2023-01-25T04:21:45.026434+0000","last_clean":"2023-01-25T04:21:43.018310+0000","last_became_active":"2023-01-25T04:21:45.026371+0000","last_became_peered":"2023-01-25T04:21:45.026371+0000","last_unstale":"2023-01-25T04:21:45.026434+0000","last_undegraded":"2023-01-25T04:21:45.026434+0000","last_fullsized":"2023-01-25T04:21:45.025824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:59:23.524854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":50,"state":"stale+active+clean","last_fresh":"2023-01-25T04:21:42.797232+0000","last_change":"2023-01-25T04:21:38.011574+0000","last_active":"2023-01-25T04:21:42.797232+0000","last_peered":"2023-01-25T04:21:42.797232+0000","last_clean":"2023-01-25T04:21:42.797232+0000","last_became_active":"2023-01-25T04:21:38.011325+0000","last_became_peered":"2023-01-25T04:21:38.011325+0000","last_unstale":"2023-01-25T04:21:48.032952+0000","last_undegraded":"2023-01-25T04:21:42.797232+0000","last_fullsized":"2023-01-25T04:21:42.797232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:23:44.964345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+undersized","last_fresh":"2023-01-25T04:21:46.030880+0000","last_change":"2023-01-25T04:21:46.030847+0000","last_active":"2023-01-25T04:21:46.030880+0000","last_peered":"2023-01-25T04:21:46.030880+0000","last_clean":"2023-01-25T04:21:44.702532+0000","last_became_active":"2023-01-25T04:21:46.030847+0000","last_became_peered":"2023-01-25T04:21:46.030847+0000","last_unstale":"2023-01-25T04:21:46.030880+0000","last_undegraded":"2023-01-25T04:21:46.030880+0000","last_fullsized":"2023-01-25T04:21:46.030434+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:24:15.475232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789972,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84399999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63900000000000001}]}]},{"osd":8,"up_from":45,"seq":193273528340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.499},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.23599999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.433}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.30199999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51100000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.55700000000000005}]}]},{"osd":7,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851920,"statfs":{"total":107374182400,"available":107368366080,"internally_reserved":0,"allocated":565248,"data_stored":258473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.625},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.87},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.03}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.25}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.584,"5min":12.584,"15min":12.584},"last":1.2230000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.304},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.109}]}]},{"osd":6,"up_from":45,"seq":193273528341,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.442}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74399999999999999}]}]},{"osd":1,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":0,"up_from":0,"seq":20,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.23799999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.313}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57899999999999996}]}]},{"osd":2,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":15.814,"5min":15.814,"15min":15.814},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.442},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.376}]},{"osd":6,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":15.757999999999999,"5min":15.757999999999999,"15min":15.757999999999999},"last":0.623}]},{"osd":8,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.49399999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":15.595000000000001,"5min":15.595000000000001,"15min":15.595000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.67100000000000004}]}]},{"osd":3,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.432}]},{"osd":5,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.46400000000000002}]},{"osd":8,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.73999999999999999}]}]},{"osd":4,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.749},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.96699999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.877}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.82599999999999996}]}]},{"osd":5,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.438}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.38}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.50600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.48799999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:21:50.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:51.118 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:51.120 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:51.131 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-01-25T04:21:50.039442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":57628,"kb_used_data":5772,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":1048518372,"statfs":{"total":1073741824000,"available":1073682812928,"internally_reserved":0,"allocated":5910528,"data_stored":2770956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702632+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:44.702632+0000","last_peered":"2023-01-25T04:21:44.702632+0000","last_clean":"2023-01-25T04:21:44.702632+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:44.702632+0000","last_undegraded":"2023-01-25T04:21:44.702632+0000","last_fullsized":"2023-01-25T04:21:44.702632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.023808+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:44.023808+0000","last_peered":"2023-01-25T04:21:44.023808+0000","last_clean":"2023-01-25T04:21:44.023808+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:44.023808+0000","last_undegraded":"2023-01-25T04:21:44.023808+0000","last_fullsized":"2023-01-25T04:21:44.023808+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.017542+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:42.017542+0000","last_peered":"2023-01-25T04:21:42.017542+0000","last_clean":"2023-01-25T04:21:42.017542+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:42.017542+0000","last_undegraded":"2023-01-25T04:21:42.017542+0000","last_fullsized":"2023-01-25T04:21:42.017542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702378+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:44.702378+0000","last_peered":"2023-01-25T04:21:44.702378+0000","last_clean":"2023-01-25T04:21:44.702378+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:44.702378+0000","last_undegraded":"2023-01-25T04:21:44.702378+0000","last_fullsized":"2023-01-25T04:21:44.702378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":58,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.797363+0000","last_change":"2023-01-25T04:21:38.011605+0000","last_active":"2023-01-25T04:21:42.797363+0000","last_peered":"2023-01-25T04:21:42.797363+0000","last_clean":"2023-01-25T04:21:42.797363+0000","last_became_active":"2023-01-25T04:21:38.011375+0000","last_became_peered":"2023-01-25T04:21:38.011375+0000","last_unstale":"2023-01-25T04:21:42.797363+0000","last_undegraded":"2023-01-25T04:21:42.797363+0000","last_fullsized":"2023-01-25T04:21:42.797363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:44:46.761850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+undersized+wait","last_fresh":"2023-01-25T04:21:45.026434+0000","last_change":"2023-01-25T04:21:45.026371+0000","last_active":"2023-01-25T04:21:45.026434+0000","last_peered":"2023-01-25T04:21:45.026434+0000","last_clean":"2023-01-25T04:21:43.018310+0000","last_became_active":"2023-01-25T04:21:45.026371+0000","last_became_peered":"2023-01-25T04:21:45.026371+0000","last_unstale":"2023-01-25T04:21:45.026434+0000","last_undegraded":"2023-01-25T04:21:45.026434+0000","last_fullsized":"2023-01-25T04:21:45.025824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:59:23.524854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":50,"state":"stale+active+clean","last_fresh":"2023-01-25T04:21:42.797232+0000","last_change":"2023-01-25T04:21:38.011574+0000","last_active":"2023-01-25T04:21:42.797232+0000","last_peered":"2023-01-25T04:21:42.797232+0000","last_clean":"2023-01-25T04:21:42.797232+0000","last_became_active":"2023-01-25T04:21:38.011325+0000","last_became_peered":"2023-01-25T04:21:38.011325+0000","last_unstale":"2023-01-25T04:21:48.032952+0000","last_undegraded":"2023-01-25T04:21:42.797232+0000","last_fullsized":"2023-01-25T04:21:42.797232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:23:44.964345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+undersized","last_fresh":"2023-01-25T04:21:46.030880+0000","last_change":"2023-01-25T04:21:46.030847+0000","last_active":"2023-01-25T04:21:46.030880+0000","last_peered":"2023-01-25T04:21:46.030880+0000","last_clean":"2023-01-25T04:21:44.702532+0000","last_became_active":"2023-01-25T04:21:46.030847+0000","last_became_peered":"2023-01-25T04:21:46.030847+0000","last_unstale":"2023-01-25T04:21:46.030880+0000","last_undegraded":"2023-01-25T04:21:46.030880+0000","last_fullsized":"2023-01-25T04:21:46.030434+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:24:15.475232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789972,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84399999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63900000000000001}]}]},{"osd":8,"up_from":45,"seq":193273528340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.499},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.23599999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.433}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.30199999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51100000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.55700000000000005}]}]},{"osd":7,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851920,"statfs":{"total":107374182400,"available":107368366080,"internally_reserved":0,"allocated":565248,"data_stored":258473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.625},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.87},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.03}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.25}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.584,"5min":12.584,"15min":12.584},"last":1.2230000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.304},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.109}]}]},{"osd":6,"up_from":45,"seq":193273528341,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.442}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74399999999999999}]}]},{"osd":1,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":0,"up_from":0,"seq":20,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.23799999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.313}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57899999999999996}]}]},{"osd":2,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":15.814,"5min":15.814,"15min":15.814},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.442},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.376}]},{"osd":6,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":15.757999999999999,"5min":15.757999999999999,"15min":15.757999999999999},"last":0.623}]},{"osd":8,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.49399999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":15.595000000000001,"5min":15.595000000000001,"15min":15.595000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.67100000000000004}]}]},{"osd":3,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.432}]},{"osd":5,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.46400000000000002}]},{"osd":8,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.73999999999999999}]}]},{"osd":4,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.749},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.96699999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.877}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.82599999999999996}]}]},{"osd":5,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.438}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.38}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.50600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.48799999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:21:51.134 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:51.457 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:51.459 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:51.471 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-01-25T04:21:50.039442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":57628,"kb_used_data":5772,"kb_used_omap":0,"kb_used_meta":51776,"kb_avail":1048518372,"statfs":{"total":1073741824000,"available":1073682812928,"internally_reserved":0,"allocated":5910528,"data_stored":2770956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53018624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":44,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702632+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:44.702632+0000","last_peered":"2023-01-25T04:21:44.702632+0000","last_clean":"2023-01-25T04:21:44.702632+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:44.702632+0000","last_undegraded":"2023-01-25T04:21:44.702632+0000","last_fullsized":"2023-01-25T04:21:44.702632+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":53,"reported_epoch":52,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.023808+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:44.023808+0000","last_peered":"2023-01-25T04:21:44.023808+0000","last_clean":"2023-01-25T04:21:44.023808+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:44.023808+0000","last_undegraded":"2023-01-25T04:21:44.023808+0000","last_fullsized":"2023-01-25T04:21:44.023808+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":40,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.017542+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:42.017542+0000","last_peered":"2023-01-25T04:21:42.017542+0000","last_clean":"2023-01-25T04:21:42.017542+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:42.017542+0000","last_undegraded":"2023-01-25T04:21:42.017542+0000","last_fullsized":"2023-01-25T04:21:42.017542+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":62,"reported_epoch":51,"state":"active+clean","last_fresh":"2023-01-25T04:21:44.702378+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:44.702378+0000","last_peered":"2023-01-25T04:21:44.702378+0000","last_clean":"2023-01-25T04:21:44.702378+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:44.702378+0000","last_undegraded":"2023-01-25T04:21:44.702378+0000","last_fullsized":"2023-01-25T04:21:44.702378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":58,"reported_epoch":50,"state":"active+clean","last_fresh":"2023-01-25T04:21:42.797363+0000","last_change":"2023-01-25T04:21:38.011605+0000","last_active":"2023-01-25T04:21:42.797363+0000","last_peered":"2023-01-25T04:21:42.797363+0000","last_clean":"2023-01-25T04:21:42.797363+0000","last_became_active":"2023-01-25T04:21:38.011375+0000","last_became_peered":"2023-01-25T04:21:38.011375+0000","last_unstale":"2023-01-25T04:21:42.797363+0000","last_undegraded":"2023-01-25T04:21:42.797363+0000","last_fullsized":"2023-01-25T04:21:42.797363+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:44:46.761850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":68,"reported_epoch":53,"state":"active+undersized+wait","last_fresh":"2023-01-25T04:21:45.026434+0000","last_change":"2023-01-25T04:21:45.026371+0000","last_active":"2023-01-25T04:21:45.026434+0000","last_peered":"2023-01-25T04:21:45.026434+0000","last_clean":"2023-01-25T04:21:43.018310+0000","last_became_active":"2023-01-25T04:21:45.026371+0000","last_became_peered":"2023-01-25T04:21:45.026371+0000","last_unstale":"2023-01-25T04:21:45.026434+0000","last_undegraded":"2023-01-25T04:21:45.026434+0000","last_fullsized":"2023-01-25T04:21:45.025824+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:59:23.524854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":49,"reported_epoch":50,"state":"stale+active+clean","last_fresh":"2023-01-25T04:21:42.797232+0000","last_change":"2023-01-25T04:21:38.011574+0000","last_active":"2023-01-25T04:21:42.797232+0000","last_peered":"2023-01-25T04:21:42.797232+0000","last_clean":"2023-01-25T04:21:42.797232+0000","last_became_active":"2023-01-25T04:21:38.011325+0000","last_became_peered":"2023-01-25T04:21:38.011325+0000","last_unstale":"2023-01-25T04:21:48.032952+0000","last_undegraded":"2023-01-25T04:21:42.797232+0000","last_fullsized":"2023-01-25T04:21:42.797232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T12:23:44.964345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":89,"reported_epoch":54,"state":"active+undersized","last_fresh":"2023-01-25T04:21:46.030880+0000","last_change":"2023-01-25T04:21:46.030847+0000","last_active":"2023-01-25T04:21:46.030880+0000","last_peered":"2023-01-25T04:21:46.030880+0000","last_clean":"2023-01-25T04:21:44.702532+0000","last_became_active":"2023-01-25T04:21:46.030847+0000","last_became_peered":"2023-01-25T04:21:46.030847+0000","last_unstale":"2023-01-25T04:21:46.030880+0000","last_undegraded":"2023-01-25T04:21:46.030880+0000","last_fullsized":"2023-01-25T04:21:46.030434+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T13:24:15.475232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9],"acting":[9],"avail_no_missing":["9"],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789972,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.62},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.82999999999999996}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.60899999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84399999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.63900000000000001}]}]},{"osd":8,"up_from":45,"seq":193273528340,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.499},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.23599999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.372},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.433}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.30199999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.60999999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51100000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.55700000000000005}]}]},{"osd":7,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5680,"kb_used_data":552,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":104851920,"statfs":{"total":107374182400,"available":107368366080,"internally_reserved":0,"allocated":565248,"data_stored":258473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.625},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.87},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":1.0780000000000001}]},{"osd":4,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":1.03}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.91400000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":1.1419999999999999},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":1.25}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":1.2010000000000001},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.584,"5min":12.584,"15min":12.584},"last":1.2230000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":1.304},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.109}]}]},{"osd":6,"up_from":45,"seq":193273528341,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.28299999999999997}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42499999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.37},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.442}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.61099999999999999}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.69599999999999995}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.65600000000000003}]},{"osd":9,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.74399999999999999}]}]},{"osd":1,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41199999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89500000000000002}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99399999999999999}]}]},{"osd":0,"up_from":0,"seq":20,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5768,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851832,"statfs":{"total":107374182400,"available":107368275968,"internally_reserved":0,"allocated":589824,"data_stored":276605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.23799999999999999}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.29799999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.67100000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.313}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.66100000000000003}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.57899999999999996}]}]},{"osd":2,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":3,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":15.814,"5min":15.814,"15min":15.814},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.52700000000000002}]},{"osd":4,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.33800000000000002}]},{"osd":5,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.442},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.376}]},{"osd":6,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":15.757999999999999,"5min":15.757999999999999,"15min":15.757999999999999},"last":0.623}]},{"osd":8,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.49399999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":15.595000000000001,"5min":15.595000000000001,"15min":15.595000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.67100000000000004}]}]},{"osd":3,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41499999999999998}]},{"osd":4,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.432}]},{"osd":5,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.54400000000000004}]},{"osd":7,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.499},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.46400000000000002}]},{"osd":8,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.77700000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.73999999999999999}]}]},{"osd":4,"up_from":45,"seq":193273528340,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5756,"kb_used_data":564,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851844,"statfs":{"total":107374182400,"available":107368288256,"internally_reserved":0,"allocated":577536,"data_stored":268869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.749},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.59399999999999997}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.67000000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.57499999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.96699999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.995},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.877}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.82599999999999996}]}]},{"osd":5,"up_from":45,"seq":193273528340,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5780,"kb_used_data":588,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851820,"statfs":{"total":107374182400,"available":107368263680,"internally_reserved":0,"allocated":602112,"data_stored":284307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.42299999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.438}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.38}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.50600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.48799999999999999}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.55400000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:21:51.473 INFO:tasks.ceph.ceph_manager.ceph:PG 1.3 is not active+clean 2023-01-25T04:21:51.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.3', 'version': "0'0", 'reported_seq': 68, 'reported_epoch': 53, 'state': 'active+undersized+wait', 'last_fresh': '2023-01-25T04:21:45.026434+0000', 'last_change': '2023-01-25T04:21:45.026371+0000', 'last_active': '2023-01-25T04:21:45.026434+0000', 'last_peered': '2023-01-25T04:21:45.026434+0000', 'last_clean': '2023-01-25T04:21:43.018310+0000', 'last_became_active': '2023-01-25T04:21:45.026371+0000', 'last_became_peered': '2023-01-25T04:21:45.026371+0000', 'last_unstale': '2023-01-25T04:21:45.026434+0000', 'last_undegraded': '2023-01-25T04:21:45.026434+0000', 'last_fullsized': '2023-01-25T04:21:45.025824+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_clean_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T06:59:23.524854+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3], 'acting': [3], 'avail_no_missing': ['3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-01-25T04:21:51.473 INFO:tasks.ceph.ceph_manager.ceph:PG 1.4 is not active+clean 2023-01-25T04:21:51.474 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.4', 'version': "0'0", 'reported_seq': 49, 'reported_epoch': 50, 'state': 'stale+active+clean', 'last_fresh': '2023-01-25T04:21:42.797232+0000', 'last_change': '2023-01-25T04:21:38.011574+0000', 'last_active': '2023-01-25T04:21:42.797232+0000', 'last_peered': '2023-01-25T04:21:42.797232+0000', 'last_clean': '2023-01-25T04:21:42.797232+0000', 'last_became_active': '2023-01-25T04:21:38.011325+0000', 'last_became_peered': '2023-01-25T04:21:38.011325+0000', 'last_unstale': '2023-01-25T04:21:48.032952+0000', 'last_undegraded': '2023-01-25T04:21:42.797232+0000', 'last_fullsized': '2023-01-25T04:21:42.797232+0000', 'mapping_epoch': 45, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_clean_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T12:23:44.964345+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-01-25T04:21:51.474 INFO:tasks.ceph.ceph_manager.ceph:PG 1.5 is not active+clean 2023-01-25T04:21:51.474 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '1.5', 'version': "0'0", 'reported_seq': 89, 'reported_epoch': 54, 'state': 'active+undersized', 'last_fresh': '2023-01-25T04:21:46.030880+0000', 'last_change': '2023-01-25T04:21:46.030847+0000', 'last_active': '2023-01-25T04:21:46.030880+0000', 'last_peered': '2023-01-25T04:21:46.030880+0000', 'last_clean': '2023-01-25T04:21:44.702532+0000', 'last_became_active': '2023-01-25T04:21:46.030847+0000', 'last_became_peered': '2023-01-25T04:21:46.030847+0000', 'last_unstale': '2023-01-25T04:21:46.030880+0000', 'last_undegraded': '2023-01-25T04:21:46.030880+0000', 'last_fullsized': '2023-01-25T04:21:46.030434+0000', 'mapping_epoch': 52, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 17, 'last_epoch_clean': 46, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'last_clean_scrub_stamp': '2023-01-25T04:20:19.062860+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-01-26T13:24:15.475232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [9], 'acting': [9], 'avail_no_missing': ['9'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 9, 'acting_primary': 9, 'purged_snaps': []} 2023-01-25T04:21:51.475 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-01-25T04:21:51.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:51.878 INFO:teuthology.orchestra.run.smithi016.stdout:{"fsid":"94576b13-ca6e-4c0d-acc4-16dd0344f762","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noout flag(s) set","count":5},"muted":false}},"mutes":[]},"election_epoch":3,"quorum":[0],"quorum_names":["a"],"quorum_age":105,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":1},"osdmap":{"epoch":58,"num_osds":10,"num_up_osds":10,"osd_up_since":1674620509,"num_in_osds":10,"osd_in_since":1674620409,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":5},{"state_name":"active+undersized","count":1},{"state_name":"active+undersized+wait","count":1},{"state_name":"stale+active+clean","count":1}],"num_pgs":8,"num_pools":1,"num_objects":0,"data_bytes":0,"bytes_used":59011072,"bytes_avail":1073682812928,"bytes_total":1073741824000},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":0,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":4,"modified":"2023-01-25T04:21:17.914625+0000","services":{}},"progress_events":{}} 2023-01-25T04:21:51.892 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2023-01-25T04:21:51.893 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:52.218 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:52.220 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:52.231 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2023-01-25T04:21:51.920633+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58560,"kb_used_data":6384,"kb_used_omap":0,"kb_used_meta":52096,"kb_avail":1048517440,"statfs":{"total":1073741824000,"available":1073681858560,"internally_reserved":0,"allocated":6537216,"data_stored":3172130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53346304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.416852+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:49.416852+0000","last_peered":"2023-01-25T04:21:49.416852+0000","last_clean":"2023-01-25T04:21:49.416852+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:49.416852+0000","last_undegraded":"2023-01-25T04:21:49.416852+0000","last_fullsized":"2023-01-25T04:21:49.416852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.320224+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:49.320224+0000","last_peered":"2023-01-25T04:21:49.320224+0000","last_clean":"2023-01-25T04:21:49.320224+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:49.320224+0000","last_undegraded":"2023-01-25T04:21:49.320224+0000","last_fullsized":"2023-01-25T04:21:49.320224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-01-25T04:21:48.097731+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:48.097731+0000","last_peered":"2023-01-25T04:21:48.097731+0000","last_clean":"2023-01-25T04:21:48.097731+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:48.097731+0000","last_undegraded":"2023-01-25T04:21:48.097731+0000","last_fullsized":"2023-01-25T04:21:48.097731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.415564+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:49.415564+0000","last_peered":"2023-01-25T04:21:49.415564+0000","last_clean":"2023-01-25T04:21:49.415564+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:49.415564+0000","last_undegraded":"2023-01-25T04:21:49.415564+0000","last_fullsized":"2023-01-25T04:21:49.415564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.043494+0000","last_change":"2023-01-25T04:21:50.043494+0000","last_active":"2023-01-25T04:21:50.043494+0000","last_peered":"2023-01-25T04:21:50.043494+0000","last_clean":"2023-01-25T04:21:50.043494+0000","last_became_active":"2023-01-25T04:21:50.043252+0000","last_became_peered":"2023-01-25T04:21:50.043252+0000","last_unstale":"2023-01-25T04:21:50.043494+0000","last_undegraded":"2023-01-25T04:21:50.043494+0000","last_fullsized":"2023-01-25T04:21:50.043494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:31:57.992525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":82,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.044010+0000","last_change":"2023-01-25T04:21:50.044010+0000","last_active":"2023-01-25T04:21:50.044010+0000","last_peered":"2023-01-25T04:21:50.044010+0000","last_clean":"2023-01-25T04:21:50.044010+0000","last_became_active":"2023-01-25T04:21:50.043743+0000","last_became_peered":"2023-01-25T04:21:50.043743+0000","last_unstale":"2023-01-25T04:21:50.044010+0000","last_undegraded":"2023-01-25T04:21:50.044010+0000","last_fullsized":"2023-01-25T04:21:50.044010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:07:16.993686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.287604+0000","last_change":"2023-01-25T04:21:50.287604+0000","last_active":"2023-01-25T04:21:50.287604+0000","last_peered":"2023-01-25T04:21:50.287604+0000","last_clean":"2023-01-25T04:21:50.287604+0000","last_became_active":"2023-01-25T04:21:50.287370+0000","last_became_peered":"2023-01-25T04:21:50.287370+0000","last_unstale":"2023-01-25T04:21:50.287604+0000","last_undegraded":"2023-01-25T04:21:50.287604+0000","last_fullsized":"2023-01-25T04:21:50.287604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:55:20.322537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.599808+0000","last_change":"2023-01-25T04:21:50.599808+0000","last_active":"2023-01-25T04:21:50.599808+0000","last_peered":"2023-01-25T04:21:50.599808+0000","last_clean":"2023-01-25T04:21:50.599808+0000","last_became_active":"2023-01-25T04:21:50.599564+0000","last_became_peered":"2023-01-25T04:21:50.599564+0000","last_unstale":"2023-01-25T04:21:50.599808+0000","last_undegraded":"2023-01-25T04:21:50.599808+0000","last_fullsized":"2023-01-25T04:21:50.599808+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:58:27.024142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.94699999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.878},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77600000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.498}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.748},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":7,"up_from":45,"seq":193273528341,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.504}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.436},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.83599999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.584,"5min":12.584,"15min":12.584},"last":0.65100000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.69399999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528342,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.626}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.502},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92400000000000004}]},{"osd":9,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.96699999999999997}]}]},{"osd":1,"up_from":57,"seq":244813135893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":0,"up_from":57,"seq":244813135893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5892,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851708,"statfs":{"total":107374182400,"available":107368148992,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.441},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66100000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.70099999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.624}]}]},{"osd":2,"up_from":45,"seq":193273528342,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":15.814,"5min":15.814,"15min":15.814},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":15.757999999999999,"5min":15.757999999999999,"15min":15.757999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.84299999999999997}]},{"osd":9,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":15.595000000000001,"5min":15.595000000000001,"15min":15.595000000000001},"last":1.073},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.70199999999999996}]}]},{"osd":3,"up_from":45,"seq":193273528341,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5892,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851708,"statfs":{"total":107374182400,"available":107368148992,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.624}]},{"osd":7,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.746}]},{"osd":9,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71399999999999997}]}]},{"osd":4,"up_from":45,"seq":193273528341,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":299645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.437}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.74299999999999999}]}]},{"osd":5,"up_from":45,"seq":193273528341,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.36099999999999999}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.621},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.46200000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.754},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:21:55.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:55.573 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:55.575 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:55.586 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-01-25T04:21:53.921004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58560,"kb_used_data":6384,"kb_used_omap":0,"kb_used_meta":52096,"kb_avail":1048517440,"statfs":{"total":1073741824000,"available":1073681858560,"internally_reserved":0,"allocated":6537216,"data_stored":3172130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53346304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.416852+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:49.416852+0000","last_peered":"2023-01-25T04:21:49.416852+0000","last_clean":"2023-01-25T04:21:49.416852+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:49.416852+0000","last_undegraded":"2023-01-25T04:21:49.416852+0000","last_fullsized":"2023-01-25T04:21:49.416852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.320224+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:49.320224+0000","last_peered":"2023-01-25T04:21:49.320224+0000","last_clean":"2023-01-25T04:21:49.320224+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:49.320224+0000","last_undegraded":"2023-01-25T04:21:49.320224+0000","last_fullsized":"2023-01-25T04:21:49.320224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-01-25T04:21:48.097731+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:48.097731+0000","last_peered":"2023-01-25T04:21:48.097731+0000","last_clean":"2023-01-25T04:21:48.097731+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:48.097731+0000","last_undegraded":"2023-01-25T04:21:48.097731+0000","last_fullsized":"2023-01-25T04:21:48.097731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.415564+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:49.415564+0000","last_peered":"2023-01-25T04:21:49.415564+0000","last_clean":"2023-01-25T04:21:49.415564+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:49.415564+0000","last_undegraded":"2023-01-25T04:21:49.415564+0000","last_fullsized":"2023-01-25T04:21:49.415564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.043494+0000","last_change":"2023-01-25T04:21:50.043494+0000","last_active":"2023-01-25T04:21:50.043494+0000","last_peered":"2023-01-25T04:21:50.043494+0000","last_clean":"2023-01-25T04:21:50.043494+0000","last_became_active":"2023-01-25T04:21:50.043252+0000","last_became_peered":"2023-01-25T04:21:50.043252+0000","last_unstale":"2023-01-25T04:21:50.043494+0000","last_undegraded":"2023-01-25T04:21:50.043494+0000","last_fullsized":"2023-01-25T04:21:50.043494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:31:57.992525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":82,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.044010+0000","last_change":"2023-01-25T04:21:50.044010+0000","last_active":"2023-01-25T04:21:50.044010+0000","last_peered":"2023-01-25T04:21:50.044010+0000","last_clean":"2023-01-25T04:21:50.044010+0000","last_became_active":"2023-01-25T04:21:50.043743+0000","last_became_peered":"2023-01-25T04:21:50.043743+0000","last_unstale":"2023-01-25T04:21:50.044010+0000","last_undegraded":"2023-01-25T04:21:50.044010+0000","last_fullsized":"2023-01-25T04:21:50.044010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:07:16.993686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.287604+0000","last_change":"2023-01-25T04:21:50.287604+0000","last_active":"2023-01-25T04:21:50.287604+0000","last_peered":"2023-01-25T04:21:50.287604+0000","last_clean":"2023-01-25T04:21:50.287604+0000","last_became_active":"2023-01-25T04:21:50.287370+0000","last_became_peered":"2023-01-25T04:21:50.287370+0000","last_unstale":"2023-01-25T04:21:50.287604+0000","last_undegraded":"2023-01-25T04:21:50.287604+0000","last_fullsized":"2023-01-25T04:21:50.287604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:55:20.322537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.599808+0000","last_change":"2023-01-25T04:21:50.599808+0000","last_active":"2023-01-25T04:21:50.599808+0000","last_peered":"2023-01-25T04:21:50.599808+0000","last_clean":"2023-01-25T04:21:50.599808+0000","last_became_active":"2023-01-25T04:21:50.599564+0000","last_became_peered":"2023-01-25T04:21:50.599564+0000","last_unstale":"2023-01-25T04:21:50.599808+0000","last_undegraded":"2023-01-25T04:21:50.599808+0000","last_fullsized":"2023-01-25T04:21:50.599808+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:58:27.024142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.94699999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.878},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77600000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.498}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.748},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":7,"up_from":45,"seq":193273528341,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.504}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.165,"5min":2.165,"15min":2.165},"last":0.57899999999999996}]},{"osd":4,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":2.0779999999999998,"5min":2.0779999999999998,"15min":2.0779999999999998},"last":0.436},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":2.2330000000000001,"5min":2.2330000000000001,"15min":2.2330000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":2.1299999999999999,"5min":2.1299999999999999,"15min":2.1299999999999999},"last":0.629},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.55000000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":2.2610000000000001,"5min":2.2610000000000001,"15min":2.2610000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.704,"5min":1.704,"15min":1.704},"last":0.83599999999999997}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":4.7690000000000001,"5min":4.7690000000000001,"15min":4.7690000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":12.584,"5min":12.584,"15min":12.584},"last":0.65100000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.593,"5min":1.593,"15min":1.593},"last":0.88200000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.69399999999999995}]}]},{"osd":6,"up_from":45,"seq":193273528342,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.626}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.502},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.181,"5min":1.181,"15min":1.181},"last":0.81399999999999995}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.92400000000000004}]},{"osd":9,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.6559999999999999,"5min":1.6559999999999999,"15min":1.6559999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.96699999999999997}]}]},{"osd":1,"up_from":57,"seq":244813135893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":0,"up_from":57,"seq":244813135893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5892,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851708,"statfs":{"total":107374182400,"available":107368148992,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.23000000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":1.546,"5min":1.546,"15min":1.546},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":2.7570000000000001,"5min":2.7570000000000001,"15min":2.7570000000000001},"last":0.68600000000000005}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.441},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":1.9059999999999999,"5min":1.9059999999999999,"15min":1.9059999999999999},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.66100000000000003}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.8520000000000001,"5min":1.8520000000000001,"15min":1.8520000000000001},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":3.153,"5min":3.153,"15min":3.153},"last":0.70099999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.9379999999999999,"5min":1.9379999999999999,"15min":1.9379999999999999},"last":0.76700000000000002}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":3.0259999999999998,"5min":3.0259999999999998,"15min":3.0259999999999998},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.81000000000000005}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.751,"15min":0.751},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.968,"5min":1.968,"15min":1.968},"last":0.624}]}]},{"osd":2,"up_from":45,"seq":193273528342,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,3,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76100000000000001}]},{"osd":3,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.234,"5min":1.234,"15min":1.234},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":15.814,"5min":15.814,"15min":15.814},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":1.48,"5min":1.48,"15min":1.48},"last":0.73399999999999999}]},{"osd":4,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.91500000000000004},{"interface":"front","average":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":15.646000000000001,"5min":15.646000000000001,"15min":15.646000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":6.96,"5min":6.96,"15min":6.96},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.55100000000000005}]},{"osd":6,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.4630000000000001,"5min":1.4630000000000001,"15min":1.4630000000000001},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":15.702999999999999,"5min":15.702999999999999,"15min":15.702999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.83299999999999996}]},{"osd":7,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.6000000000000001,"5min":1.6000000000000001,"15min":1.6000000000000001},"last":0.53500000000000003},{"interface":"front","average":{"1min":1.599,"5min":1.599,"15min":1.599},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":15.757999999999999,"5min":15.757999999999999,"15min":15.757999999999999},"last":0.85999999999999999}]},{"osd":8,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":5.4480000000000004,"5min":5.4480000000000004,"15min":5.4480000000000004},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.996,"5min":1.996,"15min":1.996},"last":0.84299999999999997}]},{"osd":9,"last update":"Wed Jan 25 04:21:18 2023","interfaces":[{"interface":"back","average":{"1min":1.552,"5min":1.552,"15min":1.552},"min":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":15.595000000000001,"5min":15.595000000000001,"15min":15.595000000000001},"last":1.073},{"interface":"front","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.411,"5min":1.411,"15min":1.411},"last":0.70199999999999996}]}]},{"osd":3,"up_from":45,"seq":193273528341,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5892,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851708,"statfs":{"total":107374182400,"available":107368148992,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[0,1,2,4,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49299999999999999}]},{"osd":2,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.45900000000000002}]},{"osd":4,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.67900000000000005}]},{"osd":5,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.624}]},{"osd":7,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70299999999999996}]},{"osd":8,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.746}]},{"osd":9,"last update":"Wed Jan 25 04:21:17 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71399999999999997}]}]},{"osd":4,"up_from":45,"seq":193273528341,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5804,"kb_used_data":612,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851796,"statfs":{"total":107374182400,"available":107368239104,"internally_reserved":0,"allocated":626688,"data_stored":299645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,5,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.437}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.63600000000000001}]},{"osd":5,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.59099999999999997}]},{"osd":6,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.52300000000000002}]},{"osd":7,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.3999999999999999,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.76200000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.577,"5min":1.577,"15min":1.577},"last":0.69799999999999995}]},{"osd":9,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":2.387,"5min":2.387,"15min":2.387},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.8069999999999999,"5min":1.8069999999999999,"15min":1.8069999999999999},"last":0.74299999999999999}]}]},{"osd":5,"up_from":45,"seq":193273528341,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,6,7,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34599999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.39200000000000002}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5620000000000001,"5min":1.5620000000000001,"15min":1.5620000000000001},"last":0.36099999999999999}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.621},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.78900000000000003}]},{"osd":8,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.6439999999999999,"5min":1.6439999999999999,"15min":1.6439999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":3.3220000000000001,"5min":3.3220000000000001,"15min":3.3220000000000001},"last":0.46200000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":0.754},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.73599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-01-25T04:21:55.589 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-01-25T04:21:55.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-01-25T04:21:55.916 INFO:teuthology.orchestra.run.smithi016.stdout: 2023-01-25T04:21:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2023-01-25T04:21:55.930 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-01-25T04:21:53.921004+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":10,"num_per_pool_osds":10,"num_per_pool_omap_osds":8,"kb":1048576000,"kb_used":58560,"kb_used_data":6384,"kb_used_omap":0,"kb_used_meta":52096,"kb_avail":1048517440,"statfs":{"total":1073741824000,"available":1073681858560,"internally_reserved":0,"allocated":6537216,"data_stored":3172130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":53346304},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":47,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.416852+0000","last_change":"2023-01-25T04:21:38.011618+0000","last_active":"2023-01-25T04:21:49.416852+0000","last_peered":"2023-01-25T04:21:49.416852+0000","last_clean":"2023-01-25T04:21:49.416852+0000","last_became_active":"2023-01-25T04:21:38.011360+0000","last_became_peered":"2023-01-25T04:21:38.011360+0000","last_unstale":"2023-01-25T04:21:49.416852+0000","last_undegraded":"2023-01-25T04:21:49.416852+0000","last_fullsized":"2023-01-25T04:21:49.416852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:52:40.604750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":57,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.320224+0000","last_change":"2023-01-25T04:21:38.014244+0000","last_active":"2023-01-25T04:21:49.320224+0000","last_peered":"2023-01-25T04:21:49.320224+0000","last_clean":"2023-01-25T04:21:49.320224+0000","last_became_active":"2023-01-25T04:21:38.013982+0000","last_became_peered":"2023-01-25T04:21:38.013982+0000","last_unstale":"2023-01-25T04:21:49.320224+0000","last_undegraded":"2023-01-25T04:21:49.320224+0000","last_fullsized":"2023-01-25T04:21:49.320224+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T05:02:36.765310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":44,"reported_epoch":56,"state":"active+clean","last_fresh":"2023-01-25T04:21:48.097731+0000","last_change":"2023-01-25T04:21:38.011284+0000","last_active":"2023-01-25T04:21:48.097731+0000","last_peered":"2023-01-25T04:21:48.097731+0000","last_clean":"2023-01-25T04:21:48.097731+0000","last_became_active":"2023-01-25T04:21:38.010943+0000","last_became_peered":"2023-01-25T04:21:38.010943+0000","last_unstale":"2023-01-25T04:21:48.097731+0000","last_undegraded":"2023-01-25T04:21:48.097731+0000","last_fullsized":"2023-01-25T04:21:48.097731+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T04:28:44.140808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":67,"reported_epoch":57,"state":"active+clean","last_fresh":"2023-01-25T04:21:49.415564+0000","last_change":"2023-01-25T04:21:38.012194+0000","last_active":"2023-01-25T04:21:49.415564+0000","last_peered":"2023-01-25T04:21:49.415564+0000","last_clean":"2023-01-25T04:21:49.415564+0000","last_became_active":"2023-01-25T04:21:38.011934+0000","last_became_peered":"2023-01-25T04:21:38.011934+0000","last_unstale":"2023-01-25T04:21:49.415564+0000","last_undegraded":"2023-01-25T04:21:49.415564+0000","last_fullsized":"2023-01-25T04:21:49.415564+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T06:45:48.010086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9],"acting":[5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":81,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.043494+0000","last_change":"2023-01-25T04:21:50.043494+0000","last_active":"2023-01-25T04:21:50.043494+0000","last_peered":"2023-01-25T04:21:50.043494+0000","last_clean":"2023-01-25T04:21:50.043494+0000","last_became_active":"2023-01-25T04:21:50.043252+0000","last_became_peered":"2023-01-25T04:21:50.043252+0000","last_unstale":"2023-01-25T04:21:50.043494+0000","last_undegraded":"2023-01-25T04:21:50.043494+0000","last_fullsized":"2023-01-25T04:21:50.043494+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:31:57.992525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":82,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.044010+0000","last_change":"2023-01-25T04:21:50.044010+0000","last_active":"2023-01-25T04:21:50.044010+0000","last_peered":"2023-01-25T04:21:50.044010+0000","last_clean":"2023-01-25T04:21:50.044010+0000","last_became_active":"2023-01-25T04:21:50.043743+0000","last_became_peered":"2023-01-25T04:21:50.043743+0000","last_unstale":"2023-01-25T04:21:50.044010+0000","last_undegraded":"2023-01-25T04:21:50.044010+0000","last_fullsized":"2023-01-25T04:21:50.044010+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T10:07:16.993686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":62,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.287604+0000","last_change":"2023-01-25T04:21:50.287604+0000","last_active":"2023-01-25T04:21:50.287604+0000","last_peered":"2023-01-25T04:21:50.287604+0000","last_clean":"2023-01-25T04:21:50.287604+0000","last_became_active":"2023-01-25T04:21:50.287370+0000","last_became_peered":"2023-01-25T04:21:50.287370+0000","last_unstale":"2023-01-25T04:21:50.287604+0000","last_undegraded":"2023-01-25T04:21:50.287604+0000","last_fullsized":"2023-01-25T04:21:50.287604+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T09:55:20.322537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":103,"reported_epoch":58,"state":"active+clean","last_fresh":"2023-01-25T04:21:50.599808+0000","last_change":"2023-01-25T04:21:50.599808+0000","last_active":"2023-01-25T04:21:50.599808+0000","last_peered":"2023-01-25T04:21:50.599808+0000","last_clean":"2023-01-25T04:21:50.599808+0000","last_became_active":"2023-01-25T04:21:50.599564+0000","last_became_peered":"2023-01-25T04:21:50.599564+0000","last_unstale":"2023-01-25T04:21:50.599808+0000","last_undegraded":"2023-01-25T04:21:50.599808+0000","last_fullsized":"2023-01-25T04:21:50.599808+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-01-25T04:20:19.062860+0000","last_clean_scrub_stamp":"2023-01-25T04:20:19.062860+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-01-26T14:58:27.024142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0],"acting":[9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9}],"osd_stats":[{"osd":9,"up_from":37,"seq":158913789973,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5904,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5248,"kb_avail":104851696,"statfs":{"total":107374182400,"available":107368136704,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5373952},"hb_peers":[2,3,4,5,6,7,8],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":2,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.85499999999999998,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.73299999999999998}]},{"osd":3,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.63}]},{"osd":4,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.54400000000000004}]},{"osd":5,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.94699999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.006,"5min":1.006,"15min":1.006},"last":0.878},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.75600000000000001}]},{"osd":8,"last update":"Wed Jan 25 04:21:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.77600000000000002}]}]},{"osd":8,"up_from":45,"seq":193273528341,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":104857600,"kb_used":5840,"kb_used_data":648,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851760,"statfs":{"total":107374182400,"available":107368202240,"internally_reserved":0,"allocated":663552,"data_stored":323425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":2,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.376},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"last":0.498}]},{"osd":3,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.58999999999999997}]},{"osd":4,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":1.117,"5min":1.117,"15min":1.117},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.81599999999999995}]},{"osd":5,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.254,"5min":1.254,"15min":1.254},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.69599999999999995}]},{"osd":7,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.748},{"interface":"front","average":{"1min":0.745,"5min":0.745,"15min":0.745},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77200000000000002}]},{"osd":9,"last update":"Wed Jan 25 04:21:15 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.71699999999999997}]}]},{"osd":7,"up_from":45,"seq":193273528341,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":104857600,"kb_used":5828,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":104851772,"statfs":{"total":107374182400,"available":107368214528,"internally_reserved":0,"allocated":651264,"data_stored":315757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[2,3,4,5,6,8,9],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56200000000000006}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":2,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":9.8079999999999998,"5min":9.8079999999999998,"15min":9.8079999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":1.2889999999999999,"5min":1.2889999999999999,"15min":1.2889999999999999},"last":0.504}]},{"osd":3,"last update":"Wed Jan 25 04:21:16 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.3